Measuring Interviewer Performance in Telephone Surveys

Abstract

In this research paper, the definition of interviewer performance is restricted to mean the capacity of an interviewer to convince a respondent to cooperate. Furthermore, it is restricted to performance in telephone surveys. To measure this aspect of performance, researchers have until now used the cooperation rate, calculated as the number of completed interviews divided by the sum of completes plus refusals, generally retaining only the first contacts made to a given phone number in order to avoid contaminating the measure with the performance of a previous interviewer. In this paper, we argue that this measure is biased because it does not allow for the measurement of performance when interviewers, usually the best ones, work on refusal conversion or on appointments. We propose a new measure of the net contribution of interviewers to the overall cooperation rate – the Net Contribution to Performance Index (NCPI) – that measures interviewer performance irrespective of the task they perform. The properties of the index are compared with the cooperation rate at first contact using a number of criteria: reliability, variability and relationship with the overall response rate, and internal and external validity. Appendix A explains how to calculate the new index.