Response Rate and Response Bias in Marketing Research
In this study, we investigate the perspectives of marketing researchers views about the two important concepts of survey research, response rate and response bias. In an attempt to answer the research questions stated, we have collected both primary and secondary data. The primary data was collected from the Academy of Marketing Science active members. Eight versions of an excerpt were taken from an actual article that was accepted for publication recently.
The first treatment was regarding the consideration of population. The subjects were manipulated with two versions of the excerpt of which one was with a Canadian sample and the second was with a so called North American sample. The second treatment was about the manipulation of the initial number of surveys sent out which as a result would change the response rate percentage. The two different versions included 500 vs. 5000 initial surveys sent out varying the response rate from 5.1% to 50.2%. The third treatment included the manipulation of Armstrong and Overton (1977) citation. First version contained a sentence that stated that the early and the late respondents were compared and no significant differences were found as evidence of no response bias including the citation of Armstrong and Overton (1977). The second version of the excerpt included a table with the expected demographics regarding the population of interest. In addition to these, the subjects were also assigned to two different conditions where they were asked to evaluate the excerpt as an author or as a reviewer. In the invitation email respondents were asked to toss a coin or to click on a web link that would toss the coin for them and select the appropriate link that corresponds with their choice. In order to assess the popular techniques of enhancing response rate, we have divided the sample into several groups as the pre-notification, the reminder and the pre-notification and reminder groups and a control group that received neither the reminder nor the pre-notification treatment. The results revealed that, according to our sample none of the techniques mentioned above, improve the response rate.
The secondary data were collected from major outlets of the marketing science (Journal of Marketing-JM, Journal of Marketing Research-JMR, and Journal of the Academy of Marketing Science-JAMS) during the periods of 2005-2010. The final sample consisted of 68 JM, 23 JMR and 84 JAMS articles. In addition to these, we also randomly selected 31 rejected articles from the Journal of Business Research (JBR) archives.
The results of the study revealed that, survey researchers do not clearly grasp the concepts of response rate and response bias. In addition, the results demonstrated that the data quality should be measured by the sample’s representativeness of the population and the researcher’s capability of decreasing the response and the non-response biases. Further, the techniques used to enhance response rate such as reminder and pre-notification letters as well as incentives are not as effective and are likely to introduce additional response bias to a study. The results also showed that the optimal data collection method researchers should consider adopting is the combination method.