Introduction

There is an extensive scientific literature documenting the persistence of racial/ethnic health inequities [1, 2]. Research suggests that the persistence of racial/ethnic health inequities in part reflects unique psychosocial stressors, such as exposure to structural, cultural, personally-mediated, and internalized racism [2, 3]. A growing body of research shows that self-reported experiences of racial discrimination adversely impact health behaviors, pre-clinical indicators of disease, physical and mental health conditions, health care utilization and mortality [4,5,6,7]. While this work has greatly improved our understanding of health inequalities, the use of population-based measures of racial discrimination to monitor and track bias to inform effective public health practices and policies remains limited.

Population-based health surveys provide important information about the health and well-being of the U.S. population and serves as the foundation for national objectives. The Behavioral Risk Factor Surveillance System (BRFSS) was designed to collect state-level data on adults to assess health-related risk behaviors, chronic health conditions, and use of preventive services for public health programming, planning, monitoring, and evaluation [8]. Data are collected by telephone interviews of representative samples within each state and consists of a standardized core questionnaire (with fixed core, rotating core, and emerging core questions consistently collected), optional modules (questions on specific topics that states elect to use), and state-added questions (questions not part of the official BRFSS questionnaire). The survey is conducted daily, by landline and cell phone, to non-institutionalized adults who are eighteen years or older. Initially piloted in 2002, the Reactions to Race module (RTRM) is optional and consists of questions assessing socially-assigned race, race consciousness, perceptions of differential treatment by race/ethnicity in healthcare and work settings, and experiences of emotional and physical symptoms resulting from race-based treatment [9]. Individual questions and composite measures utilizing the RTRM have been associated with health care utilization, preventive screenings, health behaviors, and health outcomes [10,11,12,13,14,15,16]. It is one of the few measures of racial discrimination that is captured in a state-level population-based health survey that can track and monitor racial discrimination relative to population health and guide state and local public health practices and policies to address racial health inequalities.

Since the initiation of the BRFSS, researchers have been concerned about the impact of non-response bias on the interpretation and inferences drawn from survey data [17, 18]. Studies have shown that the median response rate for the BRFSS is 48% [19]. Reasons for non-response include the method of survey administration, length of the questionnaire, and reluctance to answer sensitive questions. Additionally, sub-groups of individuals (e.g., Blacks, Hispanics, Asians, and Native Americans, low income) are more likely to be underrepresented in population-based surveys [19]. Non-response may lead to reduced power and generalizability of studying findings if those who participate are systematically different from those who do not. While response rates for US population-based surveys, such as BRFSS continue to show a steady decline (median response rate: 2011–49.7%; 2017–45.9%), assessing nonresponse bias is ever more important [20]. There may be informative differences between responders and non-responders and understanding these differences may have important implications for monitoring and taking action to address racial health inequalities. However, less attention has been given to the magnitude of nonresponse associated with the BRFSS RTRM and little is known about characteristics of responders and non-responders.

Since 2002, a total of 23 states and the District of Columbia have administered the RTRM for at least 1 year. Even fewer states administered the RTRM in consecutive years. Most recently, the South Carolina BRFSS, employed the RTRM in consecutive years (2016 and 2017). We leverage this opportunity to compare the distribution of sociodemographic variables among respondents and non-respondents to the SC-BRFSS RTRM and RTRM items. We expect to find differences in the distribution of sociodemographic variables by response type. Findings related to item-specific non-response bias could potentially identify populations under-represented in capturing experiences of and reactions to discrimination reported by the SC-BRFSS.

Methods

This study utilized data from the 2016 and 2017 South Carolina Behavioral Risk Factor Surveillance System (SC-BRFSS). The SC-BRFSS is managed by the South Carolina Department of Health and Environmental Control (SCDHEC). The survey was administered by the University of South Carolina’s Institute of Public Service and Policy Research (administration has subsequently been transferred to Winthrop University). To adjust for both sampling techniques and non-response, population weights assigned by the CDC were used (CDC, 2014). In 2016, the survey reached 11,236 respondents and in 2017, the survey reached 11,311 respondents. SC-BRFSS response rates in 2016 and 2017 were 45.8 and 47.3%, respectively, which is slightly higher than average national BRFSS response rates [21, 22]. The sample was further restricted to those who identified race/ethnicity as (Non-Hispanic) Black, Hispanic, or (Non-Hispanic) White, (with other racial/ethnic groups omitted due to small sample sizes), and who had complete interviews for the core survey and without missing data for key sociodemographic variables (i.e., education level, sex, race, age, and labor status participation), yielding a final analytic sample of 21,847 (2016: n = 10,886 respondents; 2017: n = 10,961 respondents).

The Children’s Trust of South Carolina, a nonprofit organization focused on preventing child abuse and neglect, researchers at the University of South Carolina School of Public Health, and SCDHEC partnered to finance the inclusion of the Reactions to Race Module (RTRM) in two consecutive years. RTRM, developed by the CDC Measures of Racism Work Group in 2001, is comprised of six-questions (see Table 1). Respondents are asked the RTRM questions after the core questions and towards the end of the survey. A dichotomous RTRM non-response variable was created based on respondents who were missing on all of the RTRM questions (“non-responders”) and individuals answering at least one RTRM question (“responders). RTRM item-specific non-response variable was assessed for each of the 6 RTRM questions, with individuals for whom responses were missing or who selected “refused,” considered non-responders, and those with a valid response or selected “don’t know/not sure” considered responders.

Table 1 Reactions to Race constructs included in the South Carolina Behavioral Risk Factor Surveillance System, 2016–2017
Full size table

Sociodemographic characteristics of interest included sex, age, race/ethnicity, educational attainment, income, employment status, health insurance status, and geographic residence. Age was categorized into six groups: 18–34; 35–44; 45–64; and 65+. Educational attainment was categorized as: less than high school, high school graduate, some college, and college graduate. Income was measured as: < $15,000, $15,000 to $24,999, $25,000 to $34,999, $35,000 to $49,999, and $50,000 or more. Employment status was categorized as: employed/self-employed, not employed, retired, homemaker/student; and unable to work. Health insurance status was assessed as insured and uninsured. Geographic region of residence was classified as urban and rural.

Statistical analysis

Comparing response rates across subgroups is one of the most common methods to assess non-response bias [19]. Employing similar analytic strategies as prior research [23], in 2020 estimates were generated using data from SC-BRFSS (2016, 2017) to examine non-responder distributions for the overall RTRM and the 6 RTRM questions and compared across sociodemographic characteristics. It is assumed that there is no evidence of nonresponse bias (so long as the subgroup variables are the only possible causes of response bias) if response rates are similar across subgroups; if rates are dissimilar, post-collection statistical adjustments can be made (e.g., weighting, multiple imputation) to reduce bias in population estimates resulting from nonresponse probabilities [24]. Differences between groups were examined using χ2 for all categorical variables. Logistic regression was used to determine whether factors were associated with response type adjusted for all sociodemographic variables. Sampling weights were applied to account for complex sampling design of the BRFSS. Data management and analyses were performed using SAS version 9.4 (SAS Institute Inc., Cary, NC). Institutional review board approval was not required for this study because the SC-BRFSS is a public use dataset and does not meet the criteria of human subjects research.

Results

Our sample size was comprised of 21,847 adults in the 2016 and 2017 SC-BRFSS surveys. Survey results were calculated on values using CDC’s weighting procedures and are presented in percentages in Table 2. The study sample was 52.3% female, 25.4% Black, 5.0% Hispanic, and 66.5% White respondents. More respondents graduated from high school (30.0%) than from college (24.0%). A little over a third of the sample reported making $50,000 or more a year and over half of the sample reported being employed (55.4%). Among all those completing the SC-BRFSS, 84.7% agreed to complete the 6-question RTRM (respondents) and 156.3% declined to answer the RTRM questions (non-responders).

Table 2 Distribution of key sociodemographic characteristics by response to the Reactions to Race module, South Carolina Behavioral Risk Factor Surveillance System, 2016–2017
Full size table

When comparing responders to non-responders, there were significant differences by age, race/ethnicity, income, employment status, and health insurance status. Individuals below the age of 45 tended to be non-responders in comparison to older survey respondents. Hispanics had the highest level of non-response (21.7%) in comparison to Black (16.2%) and White (14.5%) respondents. Those respondents with missing income data (21%) had higher levels of non-response compared to those with reported income. Retired respondents had the lowest level of non-response (9.3%) while homemaker and student respondents have the highest level of non-response (18.4%). Individuals who were uninsured (19.4%) had a higher percentage of non-response in comparison to those who reported health insurance (14.7%).

RTRM item-non-response is presented by race/ethnicity (Table 3). RTRM item non-response levels varied, with the highest levels of non-response observed for the questions about differential treatment in work (54.8%) and healthcare (26.9%) settings. Non-response levels for questions about socially-assigned race, race-consciousness, and emotional and physical reactions to race-based treatment ranged between 16.3–18.8%. Overall, Hispanic non-responders differed significantly from Black and White non-responders across RTRM items, with the exception of differential treatment in healthcare settings. Hispanic respondents were significantly more likely to have higher non-response to the questions about socially-assigned race, race consciousness, and emotional and physical reactions to race-based treatment in comparison to Black and White respondents. For differential treatment in work settings, Hispanic respondents (47.6%) had the lowest non-response in comparison to Black (54.5%) and White respondents (55.4%).

Table 3 Weighted1 Reactions to Race Module item-nonresponse by race/ethnicity, South Carolina Behavioral Risk Factor Surveillance System, 2016–2017
Full size table

The distribution of sociodemographic characteristics by RTRM item non-response is presented in Table 4. Across all of the RTRM items, individuals with younger age, unreported income, and uninsured were consistently overrepresented among non-responders. Differences in the distribution of sex by non-response were observed for select RTRM items. Females had higher levels of non-response to the socially-assigned race and reactions to differential treatment questions and men has higher levels of non-response to the differential treatment in work setting question.

Table 4 Distribution of sociodemographic characteristics by RTRM item non-response, South Carolina Behavioral Risk Factor Surveillance System, 2016–2017
Full size table

Table 5 shows the associations between sociodemographic characteristics and response type to the RTR module and RTRM items. When adjusting for sociodemographic characteristics, Hispanics respondents consistently had a higher odds of non-response in comparison to White respondents in the RTRM (OR = 1.36; 95% CI: 1.16, 1.61) and across all items, with the exception of differential treatment in healthcare. Respondents with unreported income had a higher odds of non-response in the RTRM (OR = 1.58; 95% CI: 1.20, 2.08). This association was generally robust across all RTRM items, with the exception of differential treatment at work (OR = 1.36; 95% CI: 0.97, 1.91). Retirees had a lower odds of non-response in the RTRM (OR = 0.75; 95% CI: 0.61, 0.92). This association was observed across all RTRM items, with the exception of differential treatment in health care, where retirees had a higher odds of non-response (OR = 1.35; 95% CI: 1.18, 1.55) in comparison to employed respondents.

Table 5 Sociodemographic factors associated with response type in South Carolina Behavioral Risk Factor Surveillance System, 2016–2017
Full size table

In sensitivity analyses, we explored the distribution of sociodemographic factors by RTRM item non-response for each racial/ethnic group (Supplementary Tables 1, 2 and 3). Overall, age, income, and insurance status were most consistently related to RTRM item non-response among Black and White respondents. Additionally among White respondents, employment status, was also consistently related to levels of non-response. Notably among Hispanics respondents, fewer sociodemographic characteristics were consistently related to RTRM item specific non-response.

Discussion

To our knowledge, this is the first study to examine sociodemographic characteristics associated with RTRM non-response using a BRFSS sample. Overall, 15.3% of survey respondents did not respond to the RTRM. Patterns of non-response varied across population subgroups. We observed heterogeneity in non-response across RTRM items where differential treatment in work (54.8%) and healthcare settings (26.9%) had the highest levels of non-response. There were significant differences in non-response by age, race/ethnicity, socioeconomic status and employment status.

In this SC-BRFSS sample, response patterns to the RTRM varied across population subgroups. Individuals who identified as Hispanic, indicated unreported income, and were uninsured had higher percentages of non-response to the RTRM, while individuals who were older and retired had lower percentages of non-response. These categories are similar to the groups that are typically non-responders to the BRFSS core. BRFSS makes efforts to weight the sample data by age, gender and race/ethnicity, with more recent administrations of the BRFSS employing a raking weighting methodology that additionally considers marital status and socioeconomic status (e.g., educational attainment and property owner/rental status) to reduce bias and improve representativeness not only in the BRFSS core but potentially in optional modules.

RTRM item-specific non-response ranged from 16.3 to 54.8%, with the highest levels of non-response observed among the questions inquiring about differential treatment in work and healthcare settings. Reasons for the high level of non-response for differential treatment in work settings may be attributed to labor market participation. In our sample, only 55.4% of the population reported being currently employed. The other portion of respondents comprised of those who reported being unemployed, retired, homemaker/student, or unable to work, and may have decided to not answer the question because of their work status. The question querying respondents about differential treatment in healthcare settings also had relatively increased levels of non-response (overall, 26.9%). We do not suspect this is a function of health insurance status since many respondents reported having some type of health insurance. However, research assessing specific psychometric properties of these questions may be needed to further refine this important question about one’s lived experience.

Racial/ethnic differences were observed in item-specific non-response. In general, Black and White respondents had relatively similar levels of non-response across RTRM items. Hispanic respondents had the lowest levels of non-response for the questions about differential treatment in work settings. Labor market participation and age of Hispanic respondents may partially explain these differences. Moreover, Hispanic respondents had the highest levels of non-response to the questions about socially-assigned race, race-consciousness, and emotional and physical reactions to race-based treatment. Willingness and motivation to respond to the aforementioned questions may be dictated by several factors. Hispanics respondents may experience a unique constellation of structural disadvantages (e.g., immigration legal status and language proficiency) that intersect with the traditional axis of inequality that emphasizes race which may influence actual and perceived encounters with negative and differential treatment [25]. A higher non-response to these questions may also be a function of the classification of Hispanics/Latinos as an ethnicity rather than a racial group. Prior research documents that directives established by the Office of Management and Budget (OMB), which mandates the standards and provides guidance for the collection of race and ethnicity data in the US, may not provide relevant options for describing Hispanic racial identity and the options can be confusing because it may not cohere with Hispanics’ understanding of race [26, 27]. The extent to which differential responses to these questions and for people of different racial background, ethnic identity, or nativity, which are all important correlates of one’s racialized lived experience, is not clear and needs to be further explored. It is possible that RTRM items carry sociopolitical implications and can evoke associations, feelings, or different judgments in the minds of respondents that give rise to particular interpretations or function differently by race/ethnicity [28]. Notably, Hispanics only represented 5% of respondents, thus, the observed levels of non-response need to be interpreted with caution. Future studies should assess differential item functioning of the RTRM items by race/ethnicity with sufficient sample sizes.

To our knowledge, BRFSS does not report response rates of optional modules in the annual data quality report. We are aware of one prior study that assessed non-response to a BRFSS optional module [23]. Crouch et al. examined the SC-BRFSS Adverse Childhood Exposures (ACE) optional module and documented differences between responders and non-responders by sex, age, race/ethnicity, education, income, and rurality in bivariate analyses. Overall, there were some similarities in the range of sociodemographic factors (e.g., age, race/ethnicity, and income) associated with non-response as evidenced from our bivariate analyses. However, unlike Crouch et al. (2018), we did not observe differences in response type by sex, educational attainment or geographic region of residence for the overall RTRM. Yet, for few RTRM items, sex and educational attainment were related to non-response. For example, differential treatment in work settings, females were over-represented among non-respondents. Further, we identified retirees and uninsured respondents overrepresented among overall RTRM non-responders. Moreover, in adjusted analyses, Hispanic respondents, those with unreported income, and retirees were most consistently associated with non-response in the RTRM and across RTRM items. This suggest that when studying race-based differential treatment and health, associations may be weaker among these groups and are likely to be conservative. It is not well known how non-response to BRFSS optional modules are affected by sociodemographic factors. Future studies estimating and assessing factors influencing RTRM and other optional module response levels may help to improve understanding of the extent of bias and its implication for interpreting results.

It is imperative that studies using these data consider employing analytic techniques that can accommodate patterns of missingness during the analysis stage. Typical approaches to compensate for variations in probability of selection and overall nonresponse, including weighting the sample data, may not fully address item-specific nonresponse. Prior research has shown that developing weights for a subset of respondents answering specific modules, for example, is not practical [29]. For data missing at random, considering approaches such as multiple imputation [17, 29, 30], full information maximum likelihood estimation [17, 30], and inverse probability weighting [31, 32] during the analysis stage are potential strategies to reduce bias and improve validity of the estimates. Further, RTRM nonresponse may also be a function of order effects. In South Carolina, the optional module is administered after core component questions and are often placed towards the end of the questionnaire.

As with any study using the BRFSS, we acknowledge that the data is limited by its retrospective, self-reported nature. Similar to questions about sensitive and stigmatizing life circumstances (e.g., abuse/neglect,) or health conditions (e.g., mental health, substance use), responding to experiences of race-based differential treatment may be subjected to recall and social desirability bias. While these results may be comparable to other states in the American South, they are not generalizable nationally given the limited sample size of Asians, Native Americans, and Hawaiian/Pacific Islanders. The RTRM was developed specifically for public health surveillance of racialized lived experiences in a US context. Despite these limitations, our paper has some important implications. Valid and reliable measures of experiences of racial discrimination in population-based surveys may be critical for guiding anti-racist action in public health practice and programs and informing anti-racist, data-driven policy implementation. This public health surveillance data is valuable for illuminating sources of inequities between specific populations. Additional research is needed to consider whether the observed factors influencing non-response in RTRM are consistent across other states with different proportions of Hispanics, Asians, and Native Americans, and multiracial populations groups. However, to provide this type of insight, it is incumbent among more states to administer this optional module as part of the BRFSS to provide additional insight regarding the contribution of racial discrimination to health and well-being.

Conclusions

We observed that responders and non-responders to the RTRM and items varied across key sociodemographic variables. Examining patterns of differential non-response could provide insight towards our understanding of estimates and trends of self-reported discrimination and race-related differential treatment and associations with health outcomes collected in public health population-based monitoring and surveillance systems. Public health surveys such as the BRFSS are essential tools for identifying and monitoring population health inequalities. However, tools for collecting, monitoring, and tracking racialized lived experiences at the state and local levels are limited. Enhancing the utility and generalizability of the RTRM across diverse groups has the potential to serve as a state-wide population-based tool to understand the relationship between population health and self-reported racial discrimination in different settings, inform data-driven population-based practices and policies for improving health promotion, disease prevention and management, and evaluate progress towards achieving health equity through avoidable inequalities, historical, and contemporary injustices [28].