Many probability sample surveys begin with a series of questions intended to elicit rosters of all household members at an address so that a random selection can be made among these members. Other surveys use roster techniques as the basis for determining which household members need to answer the survey questions and which do not qualify as respondents, while still others generate rosters in order to help respondents more accurately count the people living at the sample address. This chapter generalizes across surveys with different goals, but makes the case that an accurate roster is necessary for most survey goals.

Commonly used rostering approaches, typically based on who “usually” lives or stays at a residence, are generally assumed to be effective for their purpose, but they have some problems. The rules used to determine who should be considered a household member, and who should not, are remarkably unstandardized across U.S Census Bureau surveys and most other large surveys that utilize these procedures. For example, the Census Bureau employs different rules in different surveys. Even within a single survey, different instructions for interviewers and respondents sometimes contradict one another. However, these inconsistencies have often been warranted, due to the differing goals among surveys. In some cases, differences among rules for determining who should be counted as a resident in a household may be sensible, depending on the purpose of, or the constraints faced by, certain surveys. Thus, household rostering is an arena in which new conceptual and operational research is warranted and could help survey researchers to optimize a procedure that is used across a large number of surveys, including for the decennial Census.

While determining how many people are living at a sample address seems like it should be a fairly straightforward and simple task, there are a number of problems that survey designers must anticipate when formulating these rules. For example, some people may not live at a particular address all the time, such as retirees who spend summers and winters living in different states, or college students who may live at school for part or most of the year.

Most research on rostering has been based on the methodology used by four major surveys conducted by the U.S. Census Bureau: the Decennial Census, the American Community Survey (ACS), the Survey of Income and Program Participation, and the Current Population Survey (e.g. National Research Council, 2006). Although these surveys are all presumably designed to ascertain equivalent lists of all people living at each selected address (with each person being assigned to only one address, to prevent double-counting in principle or in practice), these four surveys employ different procedures and, therefore, seem likely to yield different results. More importantly, each individual survey occasionally offers instructions to respondents and interviewers that are logically inconsistent with one another, and the instructions sometimes entail the use of terms and concepts that are not sufficiently or clearly defined to allow respondents to easily and uniformly interpret the instructions and then comply with them whilst generating their responses. Research is needed that directly compares the outcomes from different approaches to generating a household roster so that we can assess whether or not different approaches aimed at achieving the same result are actually effective, or whether key differences arise in the rosters drawn under these different methods.

One of the major issues that has been identified with current practice in household rostering is the presence of inconsistencies with respect to the date or dates used as temporal reference points for determining whom should be listed as a resident. For example, for the 2010 Decennial Census, respondents who were having trouble filling out the paper form had the option to call in and provide their responses to the Census over the telephone. Some instructions for interviewers tell them to count people who were residing in the household on April 1, 2010, such as: “count the people living in this house, apartment or mobile home on April 1, 2010.” But then in another portion of the same set of instructions, the language changes to become less specific, instructing the interviewer, for example to “indicate foster children if they were staying at the address on or around April 1.” In the latter example, the modification to include dates around the target date is a significant departure from the original instruction. The ACS contains similar inconsistencies in its set of instructions for generating the roster. In one section, the instruction to respondents says to “Include everyone who is living or staying here for more than two months, include yourself if you are living here for more than two months, including anyone else staying here who does not have anywhere else to stay, even if they are here for less than two months.” Thus, in a single section, the instructions provide contradictory directions to the respondents for which people they should include on the roster.

Another common area of roster-related ambiguity involves how a survey should enumerate college students and military personnel. For example, the 2010 Decennial Census instructions indicated that “college students and armed forces personnel should be listed where they live and sleep most of the time.” But “most of the time” is never defined, leaving the respondent (and/or interviewer depending on the mode of administration) to arbitrarily determine who should be counted. Similarly, in the 2011 ACS, Interviewers using the Computer-Assisted Personal Interview mode were directed to read the following sequential list of instructions and questions to respondents:

  • I am going to be asking some questions about everyone who is living or staying at this address. First let’s create a list of the people, starting with you.

  • Are any of these people away now for more than two months, like a college student or someone living in the military?

  • Is there anyone else staying here even for a short time, such as a friend or relative?

  • Do any of these people have some other place where they usually stay?

The aforementioned instructions give no temporal reference point or any definitions for what constitutes these rather vague concepts of time period. That is, there is no clearly defined set of time-based metrics that the interviewer or respondent can use to determine what a “short time” or “usually stay” means. These terms could mean very different things to many respondents to a particular survey, leading to differences in the final inclusion or exclusion of individuals in the resulting household roster.

Other problems exist beyond inconsistency issues. One example is that instructions to respondents about whom to count and whom to exclude are often vague. Additionally, some survey instructions are intentionally designed as a feature of the instrument that is only seen by interviewers and never shown to respondents during the survey interview. From a methodological standpoint, this asymmetry in availability of rostering information could impact data quality. From a human factors and usability standpoint, the additional context found in these interviewer instructions could be extremely helpful for respondents while they are answering the roster questions. Another common issue is that household rostering procedures are often unnecessarily complicated and include complicated branching patterns, which increases the opportunity for mistakes.

Roster complexity is an important, although often overlooked, contributor to the relative ease of use of a survey instrument and is a concept that warrants in-depth research. American households and living situations can be very complex. Rostering rules typically attempt to account for this complexity by providing instructions to interviewers and respondents for how to accurately determine whom to actually count as a member of the household. There are many living situations that increase the difficulty of building household rosters accurately according to the given set of residence rules, including the following common issues, which do not reflect a complete set of the diverse circumstances represented across American households:

Complex households

  • Large households, which may or may not include members of extended families.

  • Tenuous attachment (Tourangeau 1993).

  • Roommates.

  • Roomers and boarders (Hainer et al. 1988; McKay 1992).

  • Students who attend boarding school.

  • College students.

  • Commuters who may or may not have a second residence to be closer to their place of work Monday–Friday.

  • Babies, whom certain respondents tend to exclude from household rosters.

  • Households where there children in a shared custody arrangement where the children are not at the sample residence every day, but might be considered as usually living or staying there by one or both parents.

  • People temporarily living away from the sample address for a variety of reasons, either in their own second residence or a residence not owned by them.

  • Concealment of household members due to respondents’ fear of losing their welfare benefits if the government discovers that the additional person or people usually live or stay at the sample address. Respondents who are immigrants, especially those containing household members who are illegally residing in the United States may also fear deportation, arrest, or other serious consequences that they believe are associated with becoming linked to an identifiable address or residence (Hainer et al. 1988; McKay 1992).

  • Homelessness, either temporary or permanent.

  • New categories, such as “couch surfers” who find residences where they can sleep on a couch, usually for a short time period, and who usually go online and use the Internet (e.g., Web sites such as Craigslist.com), to locate amenable couch-owning residences.

One common approach to addressing these challenges to accurate household rostering has been to use equally complex systems of rules, the goal of which is to determine who should count as a member of the household. However, a major drawback to this approach, especially for researchers hoping to compare data between surveys, is that these rules are not standardized in terms of content or structure. The same lack of consistency can be found across surveys if one examines the definitions provided for important terms and concepts contained within the survey questions. For example, the concept of “usual residence” is ubiquitous in rostering questions and can seem like a relatively simple concept upon initial consideration. However, consider the wide variety of methods that are employed in the process of determining whether an address is the usual residence for a generic individual named Joe:

  • Does Joe contribute money for rent, food, bills, or anything else?

  • How frequently does Joe sleep here?

  • Does Joe use this address to receive mail or phone messages?

  • Does Joe usually eat here?

  • Do you consider Joe a member of the household?

  • Does Joe have another place/places where he usually stays?

  • Does Joe perform chores like cleaning?

  • Does Joe have a say in household rules?

  • Does Joe usually live or stay here?

Compared to the large number of different ways that someone can be considered a member of the household, there is a proportionally small body of research that has examined at whether complex living situations have a significant on response tendencies and on overall data quality. Although it is possible that incorporating complex rostering rules into the design of a survey is one solution to the challenges presented by complex households, there simply has not been enough research conducted in order to draw this conclusion. Many programs of extensive empirical research are sorely needed in order to inform survey designers and researchers’ approach to conducting household rostering.

Additional rostering topics that similarly want for further research include a line of experiments aimed at determining best practices for question branching and for identifying ways to reduce the cognitive and time-related burden, for both respondents and interviewers, associated with conducting or responding to interview questions that ask respondents to apply residence rules to generate some form of a roster. Additionally, more research on self-response rostering is also needed so that researchers may gain a clearer understanding about the impact that a rostering approach may have on the survey’s data quality (instead of simply making assumptions about what the impact might be). Further, the opportunity to utilize a convergence of scientific evidence on which to base decisions about rostering approaches is absent from the corpus of survey methodology research. Specifically, much of the data that we do have about collecting roster-related data from hard-to-reach cases of highly complex living situations, and populations that were at high risk for Census under coverage, comes from a single survey, the Living Situation Survey, which was last conducted in 1993 (Schwede, L., 1993; Schwede, Blumberg, & Chan, 2005). Revisiting this targeted approach to understanding living situations in the present time period could provide a great deal of direction for designers of large surveys when they are determining how their surveys should approach household rostering.

Areas for future research:

  • Identifying the effects of using different rules in different surveys.

    • Even within the same survey, different instructions to interviewers and respondents sometimes contradict one another. What are the effects of this inconsistency?

    • Do different approaches yield different results in terms of accuracy? Response rate? Data quality? What specific impact does rostering approach have

      • • On response rates?

      • • On data quality?

      • • On interviewer and respondent satisfaction with the interview interaction itself?

      • • On the respondents’ opinion of the agency sponsoring the survey? Has it improved, declined, or stayed the same based on their experience with the interview? Does the direction or magnitude of the change seem to be related to the mode in which the survey was conducted?

  • Whether the ways that surveys operationalize usual residence are consistent with respondent notions of usual residence.

  • Which basic residence rules make the most sense and are easiest for respondents to understand?

  • What is the optimal wording for the rostering questions so that they are easy for the interviewer to consistently read, pronounce, and annunciate?

  • The literature lacks a set of strong conclusions based on large-scale, empirical investigations of de jure (e.g., a rule-based method for rostering meant to count a person his or her legal residence is, regardless of where they happened to be staying when the survey was conducted) versus de facto (e.g., location-based method based on where the person was at the time of the survey) types of residence rules.

  • The impact of newly introduced technological tools and methods of analysis available online and on mobile devices for rostering purposes (overlays, pop-ups, etc.) needs to be objectively assessed.

Ultimately, there are many avenues of research that have not been conducted to date related to the concept of household rostering. We need to gather and analyze more empirical data information in order to keep working toward a more accurated and efficient approach to the design, application, and presentation of residence rules might be and whether the same set of rules that makes sense to people also helps generate a more accurate head count.