Surveys are a tool used in everyday usability testing to gather data and confirm findings amongst target populations; however, usability researchers must be aware of biases which can be introduced through certain types of survey questions. Survey instruments are a popular way to quickly gather qualitative and quantitative data from target populations for use in usability studies. These instruments may, at times, introduce unforeseen bias, and information participants may be unaware of when questions are triangulated and results combined during the analysis phase.
Certain populations may be hesitant to answer certain questions which may divulge immigrations status, sexual orientation, marriage status, age, and socio-economic status to name a few. Of even greater concern occurs when answers to these, and other, questions are triangulated and combined to form inferences with regard to the population. For example, imagine the survey instrument contained questions regarding sexual orientation, marital status, and income levels – when triangulated and analyzed, the target population could be deemed marginalized and their results deemed unnecessary to the study, when in reality this population could potentially affect the outcome of the survey had their results been included.
- Marginalized populations
- Idiosyncratic populations
- Data triangulation
- Usability testing bias