Special feature: psychological science of survey questionnaires
- 411 Downloads
This special feature focuses on the psychological science of survey questionnaires. Survey questionnaires present a set of questions to participants who with their responses will provide data to a researcher. There might be many pitfalls that should be avoided to develop a good survey questionnaire. We have focused on psychological aspects of survey questionnaires and the key elements in designing a survey questionnaire. The aim of this special issue is to draw together multiple perspectives on survey questionnaires, to highlight important theoretical and empirical insights, and to identify key priorities of new research areas. This Research Topic stands at the intersection of psychology, measurement theory, test theory, behavioral decision theory, and cognitive science.
Standard psychological measurement studies of survey questionnaires implicitly assume that the preference ranking relations of the assessed participants are maintained even if many different methods are used. This assumption is apparently self-evident when considering the measurement of a physical quantity, such as weight. For example, when weighing each of two fish that have been caught, whether comparing their weights on a balance or measuring their weights in grams, there is expected to be little difference in the order relation as to which fish is heavier, even if the scales lack accuracy to some degree (Takemura 2014).
However, psychological studies conducted in the past suggest that preference order based on judgments made in decision-making are not necessarily the same and might be reversed in some cases. This phenomenon in which the preference order is reversed because of the difference between the reactions to judgment and choice is called the preference reversal phenomenon (Tversky and Thaler 1990). This phenomenon was reported first by psychologists such as Lindman (1971) and Lichtenstein and Slovic (1971) and attributed to the methods of selection and pricing in gambles.
Takemura (2000) suggested that there are inconsistent response patterns between different questionnaire methods, and that there are ambiguous and vague response patterns in social judgment. Tourangeau et al. (2000) and Tourangeau and Yan (2007) also suggested that there are many context effects on response patterns in survey questionnaires. Since there may be inconsistencies in response patterns among different questionnaire methods for measuring the same construct, further empirical research and theoretical developments regarding the psychological processes of survey responses will be needed to develop more reliable and consistent survey methods.
Question word-order influences on covariate effects: predicting zero-sum beliefs, by Michael Smithson et al. (Australian National University)
Coping with career development: a field study measuring the influence of work communication and employee personality, by Marcus Selart and Svein Tvedt Johansen (Norwegian School of Economics)
How does response bias emerges in lengthy sequential preference judgments? by Masahiro Morii et al. (Keio University)
Respondents with low motivation tend to choose middle category: survey questions on happiness in Japan, by Shinya Masuda, et al. (Keio University).
Smithson et al. (2017) and Selart and Johansen (2017) were invited, and Morii et al. (2017) and Masuda et al. (2017) were general submissions. The invited papers were reviewed by the editorial board to elicit suggestions and possible improvements in presentation. The general submission papers were reviewed by two or three reviewers through an ordinary review process.
In the first paper, Smithson et al. reported two studies investigating personality traits as predictors of zero-sum endorsements in questionnaires, and whether their predictive performance is moderated by statement permutations. They reported that psychopathy scale was positively associated with zero-sum endorsement whereas none of the Big-5 personality factors were. They concluded with a discussion of the methodological implications of attitude statement permutation effects.
In the second paper, Selart and Johansen tested a model of the relationship between work performance, employee personality, and perceived career development. The model hypothesized that employee personality (extraversion) modifies the relationship between work characteristics (task, diversity, work feedback) and perceived career development. Their results revealed that work characteristics were positively related to perceived career opportunities and that extraversion positively moderated the relation between work characteristics and perceived career opportunities.
They discussed an implication of the findings to survey questionnaire research.
In the third paper, Morii and Sakagami analyzed how response bias arises in lengthy preference judgments. Respondents participated in lengthy sequential preference judgments. They reported that mean preference scores increased over the first 10–20 trials, then, gradually decreased from the middle to the last trial, and that participants tended to produce the maximum and minimum scores during early trials. They demonstrated that response bias can be a function of representation order.
In the last paper, Masuda et al. investigated reason for low proportions of Japanese people rating their lives as happy. Of their study’s respondents, about 40% failed to follow instructions for at least one of the four items. Moreover, they often chose middle categories for well-being items. When those who did not follow instructions were excluded from the complete sample, middle categories showed low or no spikes, and the proportion of respondents with high rating scores increased. Thus, they suggested that one reason for reportedly low proportions of happy people in Japan was respondents who were not diligent in reading items. This finding might be applied to survey questionnaires in other research.
These papers provide direction as to how studying psychological processes in survey questionnaires can develop effective survey questionnaire methods as well as advancing our understanding of the psychology of measurement.