Introduction

IndigenousFootnote 1 participation and achievement in education is an issue of national and international significance. Within Australia, the Review of Higher Education Access and Outcomes for Aboriginal and Torres Strait Islander People (Behrendt, Larkin, Griew, & Kelly, 2012)—often referred to as the Behrendt Review—provided a clear blueprint for investing in policies and programmes that support Indigenous pathways, transitions, participation and achievement in higher education. Yet, the review also acknowledged that:

While considerable data was available through departmental program-based reporting to monitor progress, there was not always sufficient evidence to assess the overall success or otherwise of specific programs. In some cases, there were no independent evaluations of programs for the Panel to draw on. (Behrendt et al., 2012, p. 154)

The review subsequently recommended that the Australian Government and universities work together to develop a national Aboriginal and Torres Strait Islander higher education monitoring and evaluation framework (Behrendt et al., 2012). While there have been multiple investments in Indigenous higher education since the Behrendt Review, action on this recommendation has remained elusive. Similarly, the 2015 recommendation of the Aboriginal and Torres Strait Islander Higher Education Advisory Council to develop a “performance framework” has also been ignored. Before we problematise this further, it is useful to understand the broader Indigenous evaluation landscape in Australia.

Indigenous Evaluation in Australia

There has been a strong realisation in Australia that the nature of evaluation evidence being generated in response to Indigenous policy and programme investments has generally been quite poor (Hudson, 2016, 2017; Muir & Dean, 2017). Hudson (2017, p. 13) claims “Indigenous evaluations are characterised by a lack of data and the absence of a control group, as well as an over-reliance on anecdotal evidence.” The Productivity Commission (2013) and the Australian National Audit Office (2017) have both emphasised the importance of improving evaluation in Indigenous programme and policy contexts across Australia. In response, the Australian Government Department of Prime Minister and Cabinet (DPMC) developed the Indigenous Advancement Strategy Evaluation Framework (DPMC, 2018). However, this has not been explicitly linked to strengthening evaluation in Indigenous higher education. More recently, in April 2019, the Productivity Commission appointed its first Indigenous Commissioner to drive the development of a whole of government Indigenous Evaluation Strategy, although at this stage it is unclear whether this will include a focus on higher education. As such, it is useful to understand the appetite and existing evidence base to support a more focused approach in this realm.

Indigenous Evaluation in Higher Education

The notion of incorporating Indigenous Knowledges (IKs) into evaluation in Indigenous higher education contexts dovetails neatly with scholarship about Indigenous research methods and data sovereignty (Drew, Wilks, & Wilson, 2015; Kovach, 2010; Smith, 2012; Walter, 2010, 2016). Indeed, it could help to galvanise the role of IKs as a central feature of evaluation processes in this setting. The work of Rigney (1999), Kovach (2010) and Smith (2012) is particularly influential in this regard. Yet, until recently, there has been limited receptiveness within western academic institutions to develop monitoring and evaluation processes, and indicators designed to examine the cultural responsiveness of universities, particularly in relation to the lived experiences of Indigenous students and staff. This is important within the context of the release of the Universities Australia Indigenous Strategy 2017–2020, which has helped to pave a new pathway in this regard (Universities Australia, 2017).

In Australia, Rigney (2017) has published a conceptual Design and Evaluation for Indigenisation that can guide institutional change. Additional and complementary empirically based evaluation models have also been developed (Drew et al., 2015; NCSEHE, 2018; Smith, Pollard, Robertson, & Trinidad, 2017; Smith, Pollard, Robertson, & Shalley, 2018). The emergence of these conceptual frameworks is particularly timely. For example, Indigenous governance is increasingly being (re)emphasised as an important factor in Indigenous higher education policy and programme contexts, with clear objectives to grow the Indigenous higher education professional and academic workforce, and improve cultural competence in Australian universities (Buckskin et al., 2018; Universities Australia, 2011, 2017). As we have previously concluded:

There is a clear call to action for strengthening evaluation in Indigenous higher education contexts in Australia. This will require a unified response between Indigenous scholars, government and universities. (Smith et al., 2018, p. 83)

In this chapter, we draw on empirical data from a recent study about strengthening evaluation in Indigenous higher education contexts in Australia (Smith et al., 2017a; Smith, Larkin, Yibarbuk, & Guenther, 2017b, 2018). We also refer to outcomes from a national workshop about Indigenous data sovereignty in higher education (NCSEHE, 2018). A conceptual model of promising performance parameters to strengthen Indigenous higher education monitoring and evaluation in Australia emerged from this work Smith et al., (2018) (see Fig. 7.1). This model is useful for understanding potential performance parameters in a holistic sense, in relation to:

  • Students,

  • Families and community,

  • Schools and organisations, and

  • Universities.

Fig. 7.1
figure 1

Source Smith et al. (2018, p. 71) reproduced with permission

Conceptual model of potential performance parameters to strengthen Indigenous higher education monitoring and evaluation in Australia.

Full size image

However, the relevance of such performance parameters is best understood in relation to the personal and professional narratives of those who are part of the higher education system (Smith et al., 2018). As such, we use the voices of Indigenous scholars from across Australia, to discuss the challenges and opportunities associated with evaluating the effectiveness, and measuring the impacts, of culturally responsive Indigenous-focused higher education strategies currently being implemented by universities nationally. Using a combination of thematic and framework analyses, we discuss our findings within the context of Universities Australia’s (UA) Indigenous Cultural Competency Framework (2014). In the spirit of continuous quality improvement, we demonstrate where there are clear areas of alignment with the existing framework, but also highlight areas where additional strategic emphasis is required.

Methodology

The following section describes the research methodology associated with the research about strengthening evaluation in Indigenous higher education contexts in Australia. The project was approved by the Charles Darwin University Human Research Ethics Committee on 13 February 2017 (H17005). There were two separate participant groups included in this study. This included (a) policy-makers and (b) Indigenous scholars. We have privileged the voices of the Indigenous scholars in this chapter. A brief description of each participant group and the respective selection and recruitment processes is provided below. In the context of this study, an Indigenous scholar was defined as someone who identified as Indigenous and who was working in a higher education institution within an executive (such as Deputy Vice-Chancellor or Pro Vice-Chancellor), management, strategic policy, research or in a senior academic teaching-oriented role.

Twenty-four Indigenous scholars were recruited to this study using a mixture of purposive and snowball sampling. This was initially facilitated through networks known to the primary researcher (JS). However, further recruitment was aided by presenting to the National Aboriginal and Torres Strait Islander Higher Education Consortium (NATSIHEC) caucus meeting in June 2017. Targeted efforts were made to recruit Indigenous scholars from all states and territories across Australia, and while we do not claim to have a representative sample, we did succeed in this task. In the spirit of recognising Indigenous participants as sovereign people, we acknowledge the following Indigenous language groups and regions (with permission granted to acknowledge such heritage) from across Australia were involved in this study: Worimi, Palawa, Ngugi, Birapai, Wakka Wakka, Noongar, Kungarakung, Tharwal, Kaurna, Gurindji, Narungga Ngarrindjeri, Kabi Kabi, Anaiwan, Far North Queensland, Boigu Island, Pertame (Southern Arrernte), Tugga-Gah Wiradjuri, Kokoberran and Stolen Generation.

Nine Indigenous scholars have requested their comments be attributed by name in line with recent academic data sovereignty and data ownership discussions. The other Indigenous scholars have requested to remain anonymous or indicated they do not mind if they are identified. In these instances, pseudonyms have been used (where possible, pseudonyms were selected by the participants).

Eighteen respondents provided details about their length of service in the higher education sector. These participants had worked for an average of 17 years in the higher education sector. Many also reported having been involved in a variety of national Indigenous education-focused advisory groups and committees such as NATSIHEC, National Indigenous Research and Knowledges Network (NIRAKN), Ministerial Advisory Councils and/or a state and national Aboriginal and Torres Strait Islander Education Consultative Group. Many had also been involved on a university Senates or Academic Board. This demonstrates the depth of experience among the Indigenous scholars interviewed. It also means that each respondent potentially brings their own political agenda and/or vested interest to the interview context in the way they have responded. Eight interviews were conducted face-to-face with the remaining interviews conducted via telephone.

Interviews, Coding and Analysis

A semi-structured interview format was adopted. All interviews were audio-recorded. All interviews were conducted between March and November 2017. Interviews typically lasted between 45 minutes and two and half hours. All interviews were transcribed by a professional transcription service. Written field notes were compared with transcripts for accuracy prior to coding. All participants also had the opportunity to review their transcripts prior to coding.

Coding and analysis occurred parallel to the interview process. Coding happened between May 2017 and January 2018. Computer-Assisted Qualitative Data Analysis Software (CAQDAS)—NVIVO 11—was used by JS to code the textual interview data. The coding of Indigenous scholar interviews involved an inductive approach whereby codes emerged out of the data (Thomas, 2006). These codes were then repeatedly examined and analysed for consistent themes and subthemes. The thematic analysis process occurred between August 2017 and January 2018. In November 2017, a half-day coding and analysis workshop was held with the research team, including two Indigenous researchers, to discuss and interrogate the initial coding patterns and themes. Through this process, the following enablers and drivers of Indigenous evaluation in higher education were identified (Smith et al., 2018, p. 7):

  • Incentivising cultural competence,

  • Growing Indigenous leadership,

  • Addressing white privilege and power,

  • Valuing Indigenous Knowledges and prioritising Indigenous epistemologies,

  • Embracing political challenges as opportunities,

  • Recognising sovereign rights,

  • Increasing funding and resources,

  • Leading innovative policy development, implementation and reform,

  • Investing in strategy development,

  • Investing in cultural transformation, change and quality improvement,

  • Improving Indigenous student outcomes,

  • Promoting cultural standards and accreditation,

  • Reframing curricula to explicitly incorporate Indigenous Knowledges and practices, and

  • Investing in an Indigenous workforce.

The themes italicised above (for emphasis) had an explicit cultural competence connotation. We then used the thematic analysis outcomes to undertake a further framework analysis. We did this by using the guiding principles outlined in UA’s National Best Practice Framework for Indigenous Cultural Competency in Australian Universities (Universities Australia, 2014). Framework analysis has its origins in social policy contexts in the United Kingdom (UK) and is often perceived as a pragmatic approach to real-world investigations, particularly those applied to policy research (Ritchie & Spencer, 1994; Smith & Firth, 2011; Ward et al., 2013). It involves using predetermined categories to code data. The adoption of this approach enabled a deeper analysis of Indigenous scholar narratives in relation to the challenges and opportunities associated with evaluating strategies specifically related to Indigenous cultural competence in Australian universities. This was a more deductive process, whereby the guiding principles from the framework were used as pre-identified categories to support the analysis:

  1. 1.

    Indigenous people should be actively involved in university governance and management.

  2. 2.

    All graduates of Australian universities will have the knowledge and skills necessary to interact in a culturally competent way with Indigenous communities.

  3. 3.

    University research will be conducted in a culturally competent way in partnership with Indigenous participants.

  4. 4.

    Indigenous staffing will be increased at all appointment levels and, for academic staff, across a wider variety of academic fields.

  5. 5.

    Universities will operate in partnership with their Indigenous communities and will help disseminate culturally competent practices to the wider community.

Interview data relating to Indigenous cultural competencies which did not overtly align to one of these five guiding principles was allocated to a “new theme.” We argue these “new themes” could be used by UA to expand the scope of its cultural competence framework in future revisions.

Evaluation in the Context of Recognised Indigenous Cultural Competence Guiding Principles in Australian Universities

Before presenting our analysis, it is important to understand that the concept of cultural competence within higher education institutions is defined and discussed through multiple discourses; that is, cultural competence is not a static concept. Within our research, the following terms were used by Indigenous scholars as a way to talk about “cultural competence” broadly within higher education contexts: “cultural integrity”, “cultural accountability”, “cultural leadership”, “Indigenous leadership”, “Indigenous Knowledges”, “culturally responsive”, “culturally sensitive”, “Indigenous lens/perspectives/ways”, and “cultural heritage, protection and preservation.” That is, cultural competence means different things to different people. When reflecting on UA’s Indigenous cultural competency framework, one participant claimed:

It was developed and endorsed by Universities Australia several years ago. It was done through consultation with quite a number of universities … I just think it’s a fascinating topic because, as an Aboriginal person, the cultural competency framework stuff has been theorised and conceptualised into a bit of a box, whereas for Aboriginal people, it’s actually a very visceral, real experience. And it’s very easy to interpret whether a place appears to be culturally safe, secure and competent. Having said that, the framework does capture that quite well, in terms of outlining the things that universities should and could be doing to make universities that sort of place … we don’t have to throw the baby out with the bathwater. You do have things like the National Cultural Competency Framework … We need to potentially just put the magnifying glass back on there and say, okay, what’s worked? What hasn’t? Why? And really start to bring together, I think, some of that evidence and analyse it. Using perhaps a different lens. (Amber Collins)

The following analysis aims to place a magnifying glass on cultural competence as a  lens from which to view evaluation in Indigenous higher education contexts.

Indigenous Governance and Leadership

The influence of Indigenous governance and leadership, in its various forms, is perceived as important to guide Indigenous strategy development, expenditure and outcomes in higher education contexts in Australia. However, such influence can be difficult to measure through western monitoring and evaluation processes. Therefore, news ways to track success in relation to Indigenous governance and leadership are needed:

the inclusion of Aboriginal people in decision-making around how the funds are going to be used. Whether that’s an executive position or an Indigenous executive position or whether that’s an Aboriginal committee and what are their terms of reference because again what influence does that Aboriginal committee have in actually really defining the expenditure … it wouldn’t hurt to look at that across-the-board as far as evaluating the influence and success. I think influence is the biggest thing, I think that’s an interesting evaluation discussion, like influence of community, influence of the senior positions, influence of Indigenous voice and leadership. (Leanne Holt)

An important aspect of this discussion relates to the influence of Elders, and the role they play, within higher education contexts. Many participants considered Elders to be knowledge holders, and thus experts, who are increasingly being acknowledged in universities across Australia. As one participant noted:

I pushed to set up the First Council of Elders in any university … one way of honouring both reconciliation, action plans that most universities have, and the commitment to the cultural competencies and the commitment to the Declaration of the Rights of Indigenous People and the commitment to the Aboriginal Education Strategy was to have our knowledge holders as an integral part of the scholarship of the university and it’s worked well, it’s worked really well … 14 universities now have a more comprehensive and a more robust engagement of Elders so that’s one example … Whereas before that, it was all a nice gesture, drag our people out of the woodwork to give a cultural acknowledgement or have somebody just rubber stamp something … we’ve now got a National Council of Elders … they’ve now been endorsed by WINHEC to set up the Global Indigenous Elders Alliance and it’s all about not only profiling the scholarship of Indigenous knowledge and Elder knowledge but it’s also about putting in place a process of cultural heritage, protection and preservation. (Grace)

While there is increasing evidence of Indigenous leadership positions and governance structures embedded with universities, little is known about the quality of these structures. In 2011, a report entitled “On Stony Ground: Governance and Aboriginal and Torres Strait Islander Participation in Australian Universities” was Australia’s first Indigenous-led and comprehensive attempt to measure success in relation to Indigenous governance in higher education (Moreton-Robinson, Walter, Singh, & Kimber, 2011). With the exception of narrative reporting requirements associated with the Australian Government’s Indigenous Student Success Program (ISSP), there has been no equivalent assessment of the influence of Indigenous governance and leadership since the release of the report. This warrants a much closer examination, with the potential to develop new and innovative measures with a sharper focus on the value and quality of Indigenous governance and leadership outcomes.

Culturally Competent Graduates

Universities Australia has highlighted aspirations for an increase in culturally competent graduates. While this is laudable, there is a high degree of variance across Australian universities about exactly what a culturally competent graduate looks like (Frawley, 2017). As such, there are equally diverse ways in how this can be measured. As such, The University of Sydney’s National Centre for Cultural Competence (NCCC) was established with Higher Education Participation and Partnership Program funding to provide guidance in this regard.

The restructuring of degrees to have a more explicit focus on cultural competence as a graduate attribute was frequently celebrated. However, some participants questioned whether the goal to have culturally competent graduates was achievable at the scale required:

here we are now, 30 years later, and we’re still saying most of the sector is not culturally competent. It’s culturally sensitive but it’s not culturally competent. And to say that we’re going to have cultural competency as the measure of graduates is crazy when we don’t have enough people in the sector to be able to do that and we don’t have enough change within the sector to be able to do that effectively. (Grace)

There was broad agreement among participants that the current higher education workforce (in its entirety) was not considered to be culturally competent, which made it extremely difficult to grow and foster culturally competent graduates. That is, a dual focus on building a culturally competent workforce and culturally competent graduates was required if tangible improvements were to be achieved. Further research to establish the likely qualities and subsequent indicators of (a) a culturally competent workforce, and (b) culturally competent graduates, is warranted. Once qualities have been agreed, evaluating the longitudinal impacts of cultural competence—from workforce, student and organisational perspectives—will be important.

Culturally Competent Research

The importance of culturally competent research is well recognised nationally (Universities Australia, 2014). This has been emphasised through the delivery of international Indigenous research conferences co-hosted by NATSIHEC and NIRAKN. While this is influenced by the cultural competence of individual researchers, it is equally impacted by the research environment and the respective institutional policies, protocols and strategies that researchers are expected to use. As one participant questioned:

What is the ideal construct of a university for Indigenous people? So, when will the university allow a thesis in language that is not beholden to a supervisory team that their first language is English? How does a university go about acknowledging that type of Eldership, which actually could contribute to a research higher degree—research and scholarship development within the university based on our terms? (Tracey Bunda)

Many participants echoed Tracey’s comment above. Culturally competent research relied on an organisational environment that embraced IKs and practices in new, innovative and more respectful ways. NIRAKN was often perceived to be moving this agenda forward within Australia, but not necessarily at the speed or scale required to see positive systemic change. However, this was also mediated by each university’s commitment (or lack thereof) to the development of an Indigenous Research Strategy.

Increasing the Indigenous Workforce

All participants spoke about the importance of increasing the Indigenous workforce if improved outcomes in Indigenous higher education are to be achieved. However, institutional commitments to this endeavour were extremely varied.

we’re raising Indigenous employment outcomes … that cultural competency issue, I think, comes back to that institutional commitment and change, which will drive great employment outcomes and retention … But retaining them and progressing them, I think they’re the next areas that the university and other institutions need to do a lot more work on. And that comes when they actually do a lot more of that cultural change across the university, cultural competencies in the areas, reviewing their recruitment and the retention practices and stuff. (Kathy)

It was repeatedly reinforced by participants that recruitment, retention and career progression were all important factors in building and sustaining a skilled Indigenous higher education workforce across all areas of the university. This has recently been echoed in a much more comprehensive report prepared by NATSIHEC for the Australian Government (Buckskin et al., 2018). This report has made multiple Indigenous academic workforce recommendations including a national job evaluation review of Indigenous positions, and a national workforce modelling review to comprehensively examine supply and demand issues among the Indigenous academic workforce (Buckskin et al., 2018). Another message reiterated by participants was that the employment of Indigenous people in leadership and management positions (and not just Indigenous-identified or Indigenous-focused positions) was considered to be important:

I think employing the Indigenous people at that higher level, the manager level, the executive level. That’s a huge thing that we find, and I know that that’s where the government are moving towards anyway, especially in the ISSP guidelines. (AK)

This was also perceived to be critical for enhancing monitoring and evaluation capability across universities.

Partnering with Indigenous Communities

It is critically important for universities to build trusting and respectful relationships with Indigenous students, their families and the communities to which they belong, for them to successfully engage in all facets of higher education (Smith et al., 2017b). Yet, there is insufficient evaluation evidence about which programmes and policies are most effective and why (Frawley, Smith, & Larkin, 2015). Some participants argued for a more concerted focus on qualitative monitoring and evaluation measures:

we need to look more closely at the qualitative measure of how we engage with the community. How do we provide a culturally safe and responsive learning environment for Indigenous students? (David)

Other participants focused more intently on cultural leadership and cultural responsiveness as key mechanisms for engagement with Indigenous communities:

We feed into communities that are quite dispossessed from education. We feed into high unemployment areas and we feed into communities that don’t have that tradition of higher education. So that notion of cultural leadership is really, really important [for engaging communities]. (Tracey Bunda)

all institutions, whether they be schools or universities, have to be culturally responsive and draw on the funds of knowledge that the students bring to these institutions and therefore these institutions have to adjust their institutional culture by which to make schooling and universities more culturally responsive. (Lester-Irabinna Rigney)

Evaluation in the Context of New and Emerging Indigenous Cultural Competence Guiding Principles

The following themes extend beyond the focus of the current National Best Practice Framework for Indigenous Cultural Competency in Australian Universities (Universities Australia, 2014). In the spirit of continuous quality improvement, we present these new and emerging guiding principles as a means to demonstrate potential areas of expansion when the framework is next revised.

An Explicit Indigenous Lens

Concern was raised by multiple participants that an Indigenous lens is critical to the way in which monitoring and evaluation should be approached in Indigenous higher education contexts in Australia. Indeed, non-Indigenous evaluators may not have the cultural understandings or capabilities required to analyse and interpret key factors that are an inherent part of adopting an Indigenous lens. As Leanne remarks:

sometimes there’s difference between looking through a non-Indigenous lens and looking through an Indigenous lens and actually understanding some of the other considerations, particularly the cultural considerations of our communities and our students while they’re looking at it … is it driven by community, having that understanding, is it driven by community, is it considering the cultural confidence of students? … So as a non-Aboriginal person, sometimes it’s hard to understand those considerations. (Leanne Holt)

In response to recommendations from this research, a national workshop was facilitated by National Centre for Student Equity in Higher Education (NCSEHE) to coincide with the 2018 National Cultural Competence Conference, hosted by The University of Sydney in April 2018. A key focus of this workshop was to take a deeper dive into the concept of Indigenous data sovereignty as it relates to evaluation in Indigenous higher education. This process culminated in the development of a pictorial image about good practice principles that underpin Indigenous data sovereignty (NCSEHE, 2018) (see Fig. 7.2). Such principles included, among others, fostering Indigenous leadership, recognising sovereign rights, celebrating Indigenous expertise, valuing Indigenous informed innovation, and ensuring fit-for-purpose methodologies (NCSEHE, 2018). This provides a distinctly different lens with which to conceptualise and undertake monitoring and evaluation functions, when compared to current practices.

Fig. 7.2
figure 2

Source NCSEHE (2018, p. 5) reproduced with permission. Design by Jeffrey Phillips

Towards data sovereignty: a national conversation about strengthening evaluation in Indigenous higher contexts in Australia.

Full size image

Indigenous Curricula

Akin to earlier discussion about building culturally competent graduates, it is also important to strengthen the focus on Indigenous curricula. However, concerns were raised about how best to monitor and evaluate the effectiveness of Indigenous curricula:

We’ve done stuff here at this university where we’ve mapped how many courses use Indigenous information or incorporate Indigenous information. We’ve talked with people, we’ve had training courses to help them, but actually I would like some real reporting lines on how the universities take that up. (Stephanie Gilbert)

Accountability and quality were two issues consistently raised in relation to Indigenous curricula. This was poignantly highlighted by Maggie Walter who claimed:

If you just look at those universities who run units around Indigenous-related issues, some of them can look quite good. But it depends what sort of rubbish, or not rubbish, they’re teaching. The evaluation of those, which has traditionally never been done by Aboriginal people, so some of the courses, we know, are very, very poor quality because Aboriginal students all over Australia, complain about them; and the Indigenous content is very poor. (Maggie Walter)

Quality Indigenous-focused curricula were perceived to involve strengths-based notions of Indigenous people and culture, in contrast to deficit discourses. This was perceived to provide a more positive learning environment:

as a teacher, in developing curriculum, in Higher Education we’re notorious for having what I call “the mad,” “the bad” and “the sad” of Indigenous people in curriculum … no matter how many different policies we have of engagement, and getting the students in, if we’re not making the learning environment safe for them, in terms of seeing themselves reflected positively in the curriculum, whether it be, or their community, our community, reflected in the curriculum, we’re not actually going to change very much. (Zac)

Importantly, some Indigenous scholars extended this discussion to emphasise the importance of Indigenous curricula being delivered by Indigenous academics. There was a strong sense that Indigenous academics are better equipped to deliver Indigenous curricula and that a potentially greater impact can be achieved as a result.

Do you want people to be culturally aware, putting more Aboriginal content or knowledge into curriculum? It’s great that it’s in there and sure, we are extending the body of curriculum and words about Aboriginal people in the courses or are we actually changing the mindset of students and how do you really change—assess the mindset of the student or mindset of an academic from having an Aboriginal person deliver a lecture. How do you measure it and why? (Leslie)

There was also an assertion that Indigenous students prefer to see Indigenous staff within higher education institutions.

Aboriginal people, when they go into a university, they want Indigenous courses about health and education to be laden with content about Aboriginal peoples, they want university services to be Indigenised, they want to see Aboriginal staff inside universities. (Lester-Irabinna Rigney)

Accountability

A mandate to evaluate the accountability of governments and the academy was regularly identified as an important indicator to inform positive changes in relation to Indigenous higher education. There was consensus that government accountability had generally been quite poor.

One of the reasons why this world movement of Indigenous educators has taken place with the International Board of Accreditation, World Indigenous Research Alliance, etcetera, etcetera, one of the reasons why that has occurred is to address the very issues that we’re talking about. If governments across the world aren’t going to be responsive to acknowledging that there is a need for those particular type of things [cultural standards] to be addressed, if the governments weren’t going to do it, the Indigenous people themselves have done it and done it well. (Grace)

In addition, there was a high level of scepticism towards the ability of government agencies to evaluate their own Indigenous higher education policies and programmes:

I think [DPMC staff names] need some help with setting up their evaluation framework by the looks of it because they’ve put in these things like employment and the curriculum and all those things. How are they really going to evaluate that it’s working? (Leslie)

Evaluating the cultural competence, and respective accountability, of university executives was also mentioned frequently by participants, particularly in relation to Indigenous engagement and advocating for system changes aimed at improving Indigenous higher education:

how do I build the capacity of teaching and learning Deans in all disciplines across the country? I’m running workshops around professional development on how to actually embed Indigenous content in their curriculum … And so a lot of the time, people with spheres of influence in Higher Ed are not engaging in Indigenous spaces. So, if you were to look at any of the organisation behaviour literature, that is something that—you need these people to actually create change in an organisation. (Zac)

Some also extended this discussion to question whether executives appropriately modelled behaviours in accordance with institutional strategies, inferring that a lack of accountability for those people was problematic.

Promoting Cultural Standards and Accreditation

The embedding of cultural standards into the work of universities was seen as important. The World Indigenous Nations Higher Education Consortium (WINHEC) has already developed a global Indigenous-led higher education accreditation process (Malina-Wright, Robertson, & Moeke, 2010). This accreditation is explicitly about cultural standards and protocols. The Wollotuka Institute at the University of Newcastle and the Batchelor Institute of Indigenous Tertiary Education (BIITE) are the only two tertiary providers in Australia with this accreditation. The accreditation process typically involves Indigenous scholars from across different countries rigorously assessing cultural standards from an Indigenous standpoint. This is aptly summarised by Zac:

In terms of the WINHEC evaluation and accreditation, it goes—one of the things that stands out to me around when they do the evaluations is actual presence. That Indigenous business and people are present everywhere … And it involves site visits by three academics, from—they usually get one from the Pacific, North America, one from Europe, and one from your home [country]. So, you get three different perspectives in terms of the evaluation. It’s quite rigorous in what they do … I think BIITE and the Wollotuka [Institute] at the University of Newcastle are the only two. (Zac)

It was emphasised that expectations in relation to cultural standards should be a whole-of-university venture, rather than limited to an Indigenous-specific school or service within a university.

There are elements of the [cultural] standards that you would say that—the whole institution—the university should be part of cultural competency as well. (Kathy)

At present, the current Tertiary Education Quality Standards Authority (TEQSA) standards only address Indigenous higher education in relation to the recruitment and admission of Indigenous students and education policies and practices that support Indigenous people. There are notable gaps in relation to teaching and curricula, research, and corporate governance. Reflections such as this emphasise that there could be a much stronger focus on Indigenous-framed cultural standards moving forward; and that the Indigenous higher sector has a genuine interest in working with TEQSA to advance this agenda. Indeed, NATSIHEC’s recent report talks about the need for the Commonwealth Department of Education and Training, TEQSA and NATSIHEC to work together to determine whole-of-university quality standards and accountabilities in relation to measuring the quality of Indigenous student and staff participation in universities (Buckskin et al., 2018). It also outlines that strategies to incorporate IKs within universities should be included as a measure of quality in the provision of higher education (Buckskin et al., 2018).

Providing a Culturally Safe Space/Environment

It has long been argued that the provision of culturally safe spaces is an important strategy for supporting Indigenous students to grow and thrive at university (Hall & Wilkes, 2015). Generally speaking, participants indicated that more effort was needed to appropriately evaluate the effectiveness and efficacy of culturally safe spaces and environments in higher education settings.

If this [university] is not an inclusive space, and it’s not a safe space in terms of the classrooms, walking through the corridors, walking across campuses, you know Indigenous students don’t come if they don’t feel like that that’s okay. (Stephanie Gilbert)

There was a strong sentiment that if Indigenous students felt unwelcome or that their cultural identity was threatened in any way, then the university was not providing a culturally safe environment.

You can’t offer people a culturally safe study and work environment and then expect them to compromise something about their cultural integrity to such a degree that it has no relevance at all to our people. So that’s why this whole concept of cultural competency is very critical to research, to teaching, to student supports, to engagement, to partnerships and pathways … when you think that you have to compromise your cultural integrity to either get ahead or be patted on the head or to be successful in your work —everyone has to compromise to a degree but when you compromise to the extent that you almost abort the reality of what it is like to be an Aboriginal person in this country, then that’s not integrity, that’s you selling your soul, your cultural soul, just to get ahead. And that’s a dilemma, that is a big dilemma for a lot of our people whether it’s in higher ed or health or whatever (Grace).

Conclusion

This chapter has shown that cultural competence is a foundation concept for strengthening evaluation in Indigenous higher education in Australia. The guiding principles of UA’s Indigenous Cultural Competency Framework were clearly very important to the Indigenous scholars interviewed in this research. However, they were insufficient in addressing all the issues and concerns raised by Indigenous scholars. Additional markers of cultural competence within an evaluation context could potentially include adopting an explicit Indigenous lens; championing Indigenous-focused curricula (preferably delivered by Indigenous scholars); developing explicit cultural standards that are assessed through formal accreditation processes; increasing accountability of governments and university executives to prioritise cultural competence; and providing a culturally safe place/environment in university settings.

A key challenge for moving this agenda forward is addressing the “know-do gap” (Bacchi, 2008). It is clear that Australian universities are currently struggling with the implementation and evaluation of activities aimed at promoting cultural competence within Indigenous higher education contexts. An important strategy for overcoming this barrier is to privilege Indigenous worldviews in identifying workable solutions. Indeed, Indigenous methods, concepts and standpoints could potentially revolutionise the purpose of monitoring and evaluation in Indigenous higher education, and this could be used to frame what should be monitored and evaluated, when, why, how and by whom. Such an approach should embrace the grassroots principles of Indigenous data sovereignty. This would provide a useful mechanism to build a more culturally responsive evaluation culture in universities across Australia.