Journal of Urban Health

, Volume 91, Issue 2, pp 242–255 | Cite as

Development and Initial Validation of the Observer-Rated Housing Quality Scale (OHQS) in a Multisite Trial of Housing First

  • Carol E. Adair
  • Brianna Kopp
  • Jennifer Lavoie
  • Jino Distasio
  • Stephen W. Hwang
  • Aimee Watson
  • Scott Veldhuizen
  • Katherine Chislett
  • Jijian Voronka
  • Muznah Ahmad
  • Naveed Ahmed
  • Paula Goering


Quality of housing has been shown to be related to health outcomes, including mental health and well-being, yet “objective” or observer-rated housing quality is rarely measured in housing intervention research. This may be due to a lack of standardized, reliable, and valid housing quality instruments. The objective of this research was to develop and validate the Observer-Rated Housing Quality Scale (OHQS) for use in a multisite trial of a “housing first” intervention for homeless individuals with mental illness. A list of 79 housing unit, building, and neighborhood characteristics was generated from a review of the relevant literature and three focus groups with consumers and housing service providers. The characteristics were then ranked by 47 researchers, consumers, and service providers on perceived importance, generalizability, universality of value, and evidence base. Items were then drafted, scaled (five points, half values allowed), and pretested in seven housing units and with seven raters using cognitive interviewing techniques. The draft scale was piloted in 55 housing units in Toronto and Winnipeg, Canada. Items were rated independently in each unit by two trained research assistants and a housing expert. Data were analyzed using classical psychometric approaches and intraclass correlation coefficients (ICC) for inter-rater reliability. The draft scale consisted of 34 items assessing three domains: the unit, the building, and the neighborhood. Five of 18 unit items and 3 of 7 building items displayed ceiling or floor effects and were adjusted accordingly. Internal consistency was very good (Cronbach’s alpha = 0.90 for the unit items, 0.80 for the building items, and 0.92 total (unit and building)). Percent agreement ranged from 89 to 100 % within one response scale value and 67 to 91 % within one half scale value. Inter-rater reliability was also good (ICCs were 0.87 for the unit, 0.85 for the building, and 0.93 for the total scale). Three neighborhood items (e.g., distance to transit) were found to be most efficiently rated using publicly available information. The physical quality of housing can be reliably rated by trained but nonexpert raters using the OHQS. The tool has potential for improved measurement in housing-related health research, including addressing the limitations of self-report, and may also enable documenting the quality of housing that is provided by publicly funded housing programs.


Housing quality Standardized measures Housing-related health research Public housing Homelessness Mental health 



Thanks to K. Mason, C. Kelly, and C. Issak (for assisting with ethics submissions, recruitment, and supervision of local field teams); T. Bischoff for assistance with attribute reduction; A. Ladd, D. Powell, and R. Galston for site assessments; F. Weinstock and L. Stewart (who served as our housing experts); and D. Streiner (for comments on the analysis).

We also thank Jayne Barker (2008–2011), Ph.D., and Cameron Keller (2011–present), Mental Health Commission of Canada At Home/Chez Soi National Project Leads, the National Research Team, the five site research teams, the site coordinators, and the numerous service and housing providers, as well as persons with lived experience, who have contributed to this project and the research. This research has been made possible through a financial contribution from Health Canada. The views expressed herein solely represent the authors.


  1. 1.
    Krieger J, Higgins DL. Housing and health: time again for public health action. Am J Public Health. 2002; 92(5): 758–768.PubMedCentralPubMedCrossRefGoogle Scholar
  2. 2.
    Osman LM, Ayres JG, Garden C, et al. Home warmth and health status of COPD patients. Eur J Public Health. 2008; 18(4): 399–405.PubMedCrossRefGoogle Scholar
  3. 3.
    Hwang SW, Wilkins R, Tjepkema M, et al. Mortality among residents of shelters, rooming houses, and hotels in Canada: 11 year follow-up study. BMJ. 2009; 339: b4036. doi: 10.1136/bmj.b4036.PubMedCentralPubMedCrossRefGoogle Scholar
  4. 4.
    Jacobs DE, Brown MJ, Baeder A, et al. A systematic review of housing interventions and health: introduction, methods and summary findings. J Public Health Manag Pract. 2010; 16(5): S5–S10.PubMedCrossRefGoogle Scholar
  5. 5.
    Evans GW. The built environment and mental health. J Urban Health. 2003; 80(4): 536–555.PubMedCentralPubMedCrossRefGoogle Scholar
  6. 6.
    Hwang SW, Martin RE, Tolomiczenko GS, et al. The relationship between housing conditions and health status of rooming house residents in Toronto. Can J Public Health. 2003; 94(6): 436–440.PubMedGoogle Scholar
  7. 7.
    Suglia SF, Duarte CS, Sandel MT. Housing quality, housing instability and maternal mental health. J Urban Health. 2011; 88(6): 1105–1116.PubMedCentralPubMedCrossRefGoogle Scholar
  8. 8.
    Harkness J, Newman SJ, Salkever D. The cost-effectiveness of independent housing for the chronically mentally ill: do housing and neighborhood features matter? Health Serv Res. 2004; 39(5): 1341–1360.PubMedCentralPubMedCrossRefGoogle Scholar
  9. 9.
    Wells NM, Harris JD. Housing quality, psychological distress, and the mediating role of social withdrawal: a longitudinal study of low-income women. J Environ Psychol. 2007; 27: 69–78.CrossRefGoogle Scholar
  10. 10.
    Lindberg RA, Shenassa ED, Acevedo-Garcia D, et al. Housing interventions at the neighborhood level and health: a review of the evidence. J Public Health Manag Pract. 2010; 16(5): S44–S52.PubMedCrossRefGoogle Scholar
  11. 11.
    Toomey TL, Erickson DJ, Carlin BP, et al. The association between density of alcohol establishments and violent crime within urban neighborhoods. Alcohol Clin Exp Res. 2012. doi: 10.1111/j.1530-0277.2012.01753.x.PubMedCentralPubMedGoogle Scholar
  12. 12.
    Guite HF, Clark C, Ackrill G. The impact of the physical and urban environment on mental well-being. Public Health. 2006; 120(12): 1117–1126.PubMedCrossRefGoogle Scholar
  13. 13.
    Matheson FI, Moineddin R, Dunn JR, et al. Urban neighborhoods, chronic stress, gender and depression. Soc Sci Med. 2006; 63: 2604–2616.PubMedCrossRefGoogle Scholar
  14. 14.
    Ludwig J, Duncan GJ, Gennetian LA, et al. Neighborhood effects on the long-term well-being of low-income adults. Sci. 2012; 337: 1505.CrossRefGoogle Scholar
  15. 15.
    Wright PA, Kloos B. Housing environment and mental health outcomes: a levels of analysis perspective. J Environ Psychol. 2007; 27(1): 79–89.PubMedCentralPubMedCrossRefGoogle Scholar
  16. 16.
    O’Campo P, Salmon C, Burke J. Neighbourhoods and mental well-being: what are the pathways? Health Place. 2009; 15: 56–68.PubMedCrossRefGoogle Scholar
  17. 17.
    Fields D. Emotional refuge? Dynamics of place and belonging among formerly homeless individuals with mental illness. Emot Space Soc. 2010. doi: 10.1016/j.emospa.2010.09.002.Google Scholar
  18. 18.
    Keall M, Baker MG, Howden-Chapman P, et al. Assessing housing quality and its impact on health, safety and sustainability. J Epidemiol Community Health. 2010; 64: 765–771.PubMedCrossRefGoogle Scholar
  19. 19.
    Tsemberis S, Gulcur L, Nakae M. Housing First, consumer choice, and harm reduction for homeless individuals with a dual diagnosis. AJPH. 2004; 94(4): 651–656. doi: 10.2105/AJPH.94.4.651.CrossRefGoogle Scholar
  20. 20.
    Goering PN, Streiner DL, Adair C, et al. The At Home/Chez Soi trial protocol: a pragmatic, multi-site, randomized controlled trial of a Housing First intervention for homeless individuals with mental illness in five Canadian cities. BMJ Open. 2011; 1: e000323. doi: 10.1136/bmjopen-2011-00323.PubMedCentralPubMedCrossRefGoogle Scholar
  21. 21.
    Tsemberis S, Rogers ES, Rodis E, et al. Housing satisfaction for persons with psychiatric disabilities. J Community Psychol. 2003; 31(6): 581–590.CrossRefGoogle Scholar
  22. 22.
    Toro PA, Bellavia CW, Daeschler CV, et al. Evaluating an intervention for homeless persons: results of a field experiment. J Consul Clin Psychol. 1997; 65(3): 476–484.CrossRefGoogle Scholar
  23. 23.
    Evans GW, Wells NM, Chan HYE, Saltzman H. Housing quality and mental health. J Consul Clin Psychol. 2000; 68(3): 526–530.CrossRefGoogle Scholar
  24. 24.
    US DHHS, FDA, CDER, CBER, CDRH. Guidance for industry: patient-reported outcome measures: use in medical product development to support labeling claims. December 2009.Google Scholar
  25. 25.
    Clark LA, Watson D. Constructing validity: basic issues in objective scale development. Psychol Assess. 1995; 7(3): 309–319.CrossRefGoogle Scholar
  26. 26.
    Streiner DL, Norman GR. Health measurement scales: a practical guide to their development and use. 4th ed. Oxford: Oxford University Press; 2008.CrossRefGoogle Scholar
  27. 27.
    McColl E, Meadows K, Barofsky I. Cognitive aspects of survey methodology and quality of life assessment. Qual Life Res. 2003; 12: 217–218.PubMedCrossRefGoogle Scholar
  28. 28.
    Willis GB. Cognitive interviewing: a tool for improving questionnaire design. Thousand Oaks, CA: Sage; 2005.Google Scholar
  29. 29.
    Collins D. Pretesting survey instruments: an overview of cognitive methods. Qual Life Res. 2003; 12: 229–238.PubMedCrossRefGoogle Scholar
  30. 30.
    Carr LJ, Dunsiger SI, Marcus BH. Validation of Walk Score for estimating access to walkable amenities. Br J Sports Med. 2010. doi: 10.1136/bjsm.2009.069609.PubMedGoogle Scholar
  31. 31.
    Canadian Mortgage and Housing Corporation. Rental market report Canada highlights, Spring 2011.Google Scholar
  32. 32.
    Statistics Canada. Census Canada 2006 Accessed 20 July 2013.

Copyright information

© The New York Academy of Medicine 2014

Authors and Affiliations

  • Carol E. Adair
    • 1
  • Brianna Kopp
    • 2
  • Jennifer Lavoie
    • 3
  • Jino Distasio
    • 4
  • Stephen W. Hwang
    • 5
  • Aimee Watson
    • 6
  • Scott Veldhuizen
    • 7
  • Katherine Chislett
    • 8
  • Jijian Voronka
    • 9
  • Muznah Ahmad
    • 5
  • Naveed Ahmed
    • 5
  • Paula Goering
    • 7
  1. 1.Departments of Psychiatry and Community Health SciencesUniversity of CalgaryCalgaryCanada
  2. 2.Mental Health Commission of CanadaCalgaryCanada
  3. 3.Department of CriminologyWilfrid Laurier UniversityBrantfordCanada
  4. 4.Institute of Urban StudiesUniversity of WinnipegWinnipegCanada
  5. 5.Centre for Research on Inner City HealthSt. Michael’s HospitalTorontoCanada
  6. 6.Mental Health Commission of CanadaTorontoCanada
  7. 7.Centre for Addiction and Mental HealthTorontoCanada
  8. 8.Community Services DepartmentRegional Municipality of NiagaraSt. CatharinesCanada
  9. 9.Department of Humanities, Social Science and Social JusticeUniversity of TorontoTorontoCanada

Personalised recommendations