Springer Nature is making SARS-CoV-2 and COVID-19 research free. View research | View latest news | Sign up for updates

Rating Health and Social Indicators for Use with Indigenous Communities: A Tool for Balancing Cultural and Scientific Utility


This study reports on the development and evaluation of a rating tool to assess the scientific utility and cultural appropriateness of community-level indicators for application with Indigenous populations. Indicator criteria proposed by the U.S. Institute of Medicine were culturally adapted through reviewing the literature and consultations with academic and Indigenous stakeholders. Pre-testing and collaborator feedback drove the iterative development of the tool with stakeholder groups in Canada, Aotearoa/New Zealand, and Australia. Pilot testing with 17 raters across countries involved rating the same selection of six health and social indicators using a six-point ordinal scale. The final version of the rating tool includes 16 questions within three criterion domains: importance, soundness, and viability. Academic and community stakeholder review established face and content validity. The indicator rating tool demonstrated good internal consistency and excellent inter-rater reliability for two of three pilot testing groups. Use of this instrument can strengthen collaborative research planning and evaluation with Indigenous communities through selection of relevant and culturally appropriate indicators for application to public health research, prevention programmes, and health and social policy.

This is a preview of subscription content, log in to check access.

Fig. 1


  1. Andersen, T., & Poppel, B. (2002). Living conditions in the Arctic. Social Indicators Research, 58, 191–216. doi:10.1023/A:1015787901370.

  2. Armitage, A. (1995). Comparing the policy of aboriginal assimilation: Australia, Canada, and New Zealand. Vancouver: UBC Press.

  3. Australian Bureau of Statistics, & Australian Institute of Health and Welfare. (2003). The health and welfare of Australia’s Aboriginal and Torres Strait Islander peoples, 2003. Canberra: Australian Bureau of Statistics.

  4. Baum, F., MacDougall, C., & Smith, D. (2006). Participatory action research. Journal of Epidemiology and Community Health, 60, 854–857. doi:10.1136/jech.2004.028662.

  5. Besleme, K., Maser, E., & Sliverstein, J. (1999). A community indicators case study: addressing the quality of life in two communities. San Francisco: Redefining Progress.

  6. Black, A., & Hughes, P. (2001). The identification and analysis of indicators of community strength and outcomes. Canberra: Department of Family and Community Services, Commonwealth of Australia.

  7. Cargo, M., & Mercer, S. (2008). The value and challenges of participatory research in public health: strengthening its practice. Annual Review of Public Health, 29, 325–350. doi:10.1146/annurev.publhealth.29.091307.083824.

  8. Cargo, M., Petersen, L., Levesque, L., & Macaulay, A. (2007). Physical activity and perceived wholistic health in indigenous youth. Pimitziwin, 5, 87–109.

  9. Cicchetti, D. V. (1994). Guidelines, criteria, and rules of thumb for evaluating normed and standardized assessment instruments in psychology. Psychological Assessment, 6, 284–290. doi:10.1037/1040-3590.6.4.284.

  10. Cunningham, J., & Beneforti, M. (2005). Investigating indicators for measuring the health and social impact of sport and recreation programs in Australian indigenous communities. International Review for the Sociology of Sport, 40, 89–98. doi:10.1177/1012690205052170.

  11. Daniel, M., Green, L. W., Marion, S. A., Gamble, D., Herbert, C. P., et al. (1999). Effectiveness of community-directed diabetes prevention and control in a rural Aboriginal population in British Columbia, Canada. Social Science and Medicine, 48, 815–832. doi:10.1016/S0277-9536(98)00403-1.

  12. Das, D., Baker, M., & Calder, L. (2006). Tuberculosis epidemiology in New Zealand: 1995–2004. The New Zealand Medical Journal, 119, 2249–2264.

  13. Durie, M. (1994). Whaiora: Maori health development. Auckland: Oxford University Press.

  14. Dyck, R. F., Klomp, H., Marciniuk, D. D., Tan, L., Stang, M. R., Ward, H. A., et al. (2007). The relationship between diabetes and tuberculosis in Saskatchewan: comparison of Registered Indians and other Saskatchewan people. Canadian Journal of Public Health. Revue Canadienne de Sante Publique, 98, 55–59.

  15. Ellison-Loschmann, L., & Pearce, N. (2006). Improving access to health care among New Zealand’s Maori population. American Journal of Public Health, 96, 612–617. doi:10.2105/AJPH.2005.070680.

  16. Fisher, P. A., & Ball, T. J. (2005). Balancing empiricism and local cultural knowledge in the design of prevention research. Journal of Urban Health, 82, iii44–iii55. doi:10.1093/jurban/jti063.

  17. Giles, B. G., & Findlay, C. S. (2004). Diabetes indicators for Aboriginal communities: final report. Ottawa: Institute of the Environment.

  18. Guthrie, J. A., Dore, G. J., McDonald, A. M., & Kaldor, J. M. (2000). HIV and AIDS in aboriginal and Torres Strait Islander Australians: 1992–1998. The National HIV Surveillance Committee. The Medical Journal of Australia, 172, 266–269.

  19. Hancock, T., Labonte, R., & Edwards, R. (1999). Indicators that count! Measuring population health at the community level. Canadian Journal of Public Health. Revue Canadienne de Sante Publique, 90(suppl. 1), S22–S26.

  20. Health Canada First Nations and Inuit Health Branch. (2003). Statistical profile on the health of First Nations in Canada. Ottawa: Health Canada.

  21. Health Research Council of New Zealand. (1998). Guidelines for researchers on health research involving Maori. Auckland: Health Research Council.

  22. Institute of Medicine, Committee on the National Quality Report on Health Care Delivery. (2001). Envisioning the national health care quality report. Washington DC: National Academies Press.

  23. Karjala, M. K., & Dewhurst, S. M. (2003). Including aboriginal issues in forest planning: a case study in central interior British Columbia, Canada. Landscape and Urban Planning, 64, 1–17. doi:10.1016/S0169-2046(02)00196-2.

  24. Karjala, M. K., Sherry, E. E., & Dewhurst, S. M. (2004). Critieria and indicators for sustainable forest planning: a framework for recording Aboriginal resource and social values. Forest Policy and Economics, 6, 95–110. doi:10.1016/S1389-9341(02)00117-X.

  25. Kingsley, T. (1999). Building and operating neighborhood indicator systems: a guidebook. Washington DC: The Urban Institute.

  26. Kramers, P. G. (2003). The ECHI project: health indicators for the European community. European Journal of Public Health, 13, 101–106. doi:10.1093/eurpub/13.suppl_1.101.

  27. Maori Health Committee of the Health Research Council of New Zealand. (1998). Guidelines for researchers on health research involving Maori, 1998. Auckland: HRCNZ.

  28. Marks, E., Cargo, M. D., & Daniel, M. (2007). Constructing a health and social indicator framework for Indigenous community health research. Social Indicators Research, 82, 93–110. doi:10.1007/s11205-006-9016-z.

  29. Martens, P. J., Sanderson, D., & Jebamani, L. S. (2005). Mortality comparisons of First Nations to all other Manitobans: a provincial population-based look at health inequalities by region and gender. Canadian Journal of Public Health. Revue Canadienne de Sante Publique, 96, S33.

  30. Medical Research Council of Canada. Natural Sciences and Engineering Research Council of Canada, & Social Sciences and Humanities Research Council of Canada. (2003). Tri-Council policy statement: ethical conduct for research involving humans. Section 6. Research involving Aboriginal peoples. Ottawa: Public Works and Government Services Canada.

  31. Natcher, D. C., & Hickey, C. G. (2002). Putting the community back into community-based resource management: a criteria and indicators approach to sustainability. Human Organization, 61, 350–363.

  32. National Health & Medical Research Council. (2003). Values and ethics: guidelines for ethical conduct in Aboriginal and Torres Strait Islander health research. Canberra: Commonwealth of Australia.

  33. Norris, T., & Pittman, M. (2000). The healthy communities movement and the coalition for healthier cities and communities. Public Health Reports, 115, 118–124. doi:10.1093/phr/115.2.118.

  34. Ontario Healthy Communities Coalition. (1999). Pathways to a healthy community: an indicators and evaluation tool kit. Toronto: Ontario Healthy Communities Coalition.

  35. Pearce, N. (1996). Traditional epidemiology, modern epidemiology, and public health. American Journal of Public Health, 86, 678–683.

  36. Popovich, M. G. (1996). Toward results-oriented intergovernmental systems: an historical look at the development of the Oregon Benchmarks. Washington DC: The Alliance for Redesigning Government of the National Academy of Public Administration.

  37. Raphael, D., Steinmetz, B., Renwick, R., Rootman, I., Brown, I., Sehdev, H., et al. (1999). The community quality of life project: a health promotion approach to understanding communities. Health Promotion International, 14, 197–210. doi:10.1093/heapro/14.3.197.

  38. Redefining Progress, & Earthday Network. (2002). Sustainability starts in your community: a community indicators guide. Oakland: Redefining Progress.

  39. Reed, M. S., & Dougill, A. J. (2002). Participatory selection process for indicators of rangeland condition in the Kalahari. The Geographical Journal, 168, 224–234. doi:10.1111/1475-4959.00050.

  40. Riddell, T. (2005). Heart failure hospitalisations and deaths in New Zealand: patterns by deprivation and ethnicity. The New Zealand Medical Journal, 118, 1254–1264.

  41. Sawicki, D. S., & Flynn, P. (1996). Neighborhood indicators: a review of the literature and an assessment of conceptual and methodological issues. Journal of the American Planning Association. American Planning Association, 62, 165–183. doi:10.1080/01944369608975683.

  42. Schnarch, B. (2004). Ownership, control, access, and possession (OCAP) of self-determination applied to research. A critical analysis of contemporary First Nations research and some options for First Nations communities. Journal of Aboriginal Health, 1, 80–95.

  43. Smith, L. T. (1999). Decolonizing methodologies: research and Indigenous peoples. New York: Zed Books.

  44. Steering Committee for the Review of Government Service Provision. (2003). Overcoming Indigenous disadvantage: key indicators, 2003. Canberra: Productivity Commission.

  45. Stein, B. (1996). The Oregon Benchmarks experience. In Canadian Council on Social Development (Ed.), Measuring Well-Being: Proceedings from a Symposium on Social Indicators: Final Report. Ottawa: CCSD.

  46. Streiner, D. L., & Norman, G. R. (2003). Health measurement scales: a practical guide to their development and use (3rd ed.). Oxford: Oxford University Press.

  47. Ten Fingers, K. (2005). Rejecting, revitalizing, and reclaiming: First Nations work to set the direction of research and policy development. Canadian Journal of Public Health. Revue Canadienne de Sante Publique, 96(suppl. 1), S60–S63.

  48. Thompson, S. J., & Gifford, S. M. (2000). Trying to keep a balance: the meaning of health and diabetes in an urban Aboriginal community. Social Science and Medicine, 51, 1457–1472. doi:10.1016/S0277-9536(00)00046-0.

  49. Tyler Norris Associates. Redefining Progress, & Sustainable Seattle. (1997). The community indicators handbook: measuring progress toward healthy and sustainable communities. Oakland: Redefining Progress.

  50. U.S. Environmental Protection Agency. Green communities. Retrieved June 2007, from http://www.epa.gov/greenkit/index.htm.

  51. VicHealth Koori Health Research and Community Development Unit. (2000). We don’t like research. But in Koori hands it could make a difference. Melbourne: Victoria.

  52. Waddell, S. (1995). Lessons from the healthy cities movement for social indicator development. Social Indicators Research, 34, 213–235. doi:10.1007/BF01079197.

  53. Walker, R., Ballard, J., & Taylor, C. (2002). Investigating appropriate evaluation methods and indicators for Indigenous housing programs. Melbourne: Australian Housing and Urban Research Institute.

  54. Walker, R., Ballard, J., & Taylor, C. (2003). Developing paradigms and discourses to establish more appropriate evaluation frameworks and indicators for housing programs. Melbourne: Australian Housing and Urban Research Institute.

  55. Wilson, K., & Rosenberg, M. W. (2002). Exploring the determinants of health for First Nations peoples in Canada: can existing frameworks accommodate traditional activities? Social Science and Medicine, 55, 2017–2031. doi:10.1016/S0277-9536(01)00342-2.

  56. Young, T. K., Reading, J., Elias, B., & O’Neil, J. D. (2000). Type 2 diabetes mellitus in Canada’s First Nations: status of an epidemic in progress. Canadian Medical Association Journal, 163, 561–566.

Download references


This project was funded by the Canadian Institutes of Health Research through the International Collaborative Indigenous Health Research Partnership program and the Global Health Research Program (#109286). MD was supported by a Canada Research Chair for Population Health. We express thanks to: Janice Muir and Tania Jones (Department of Human Services Victoria, Australia); Dr. Alf Bamblett (Victorian Aboriginal Community Services Association, Australia); Veronica Weisz, Fiona Barnes, and Petra White (Victorian Aboriginal Education Association Inc., Australia); Peter Waples-Crowe (Victorian Aboriginal Community Controlled Health Organisation, Australia); Paul Paton (Victorian Aboriginal Corporation for Languages, Australia); Joyce Doyle (Rumbalara Football Netball Club, Australia); Leah Johnston, Professor Di Bretherton, Professor Ian Anderson, and Dr. Michael Tynan (University of Melbourne, Australia); Dr. Ross Bailie, Julie Brimblecombe, Joseph Fitz, and Maria Scarlett (Menzies School of Health Research, Northern Territory, Australia); Amelia McGregor (Kahnawake School Diabetes Prevention Project, Québec, Canada); Jocelyn Bruyere (Cree Nation Tribal Health Centre, Manitoba, Canada); Dr. Sharon Bruce (University of Manitoba, Canada); Dr. T Kue Young (University of Toronto, Canada); Dr. David Dannenbaum, Jill Torrie, Solomon Awashish, and Pierre Lejeune (Cree Board of Health and Social Services of James Bay, Québec, Canada); Morehu Te Whare (Ngati Maniapoto, Aotearoa/New Zealand); and Morehu Timoti Tokomauri (Ngati Tuwharetoa, Aotearoa/New Zealand). We also express gratitude to all others involved in teleconferences, pre-testing and pilot testing sessions, including people of the Yolngu community of Galiwin’ku, Northern Territory, Australia; nga tangata o Te Arawa, Ngati Porou me Tainui waka, Aotearoa; the Cree Nation Tribal Health Centre in Manitoba, Canada; and the Cree Board of Health and Social Services of James Bay, Québec, Canada.

Author information

Correspondence to Mark Daniel.

Appendix 1

Appendix 1

Final indicator rating tool .

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Daniel, M., Cargo, M., Marks, E. et al. Rating Health and Social Indicators for Use with Indigenous Communities: A Tool for Balancing Cultural and Scientific Utility. Soc Indic Res 94, 241 (2009). https://doi.org/10.1007/s11205-008-9420-7

Download citation


  • Community participation
  • Health status indicators
  • Indigenous populations
  • Reliability and validity
  • Programme planning