1 Introduction

“Leading” or “suggestive” questions are questions that influence or cue the respondent to provide a certain response desired by the person formulating the question (Rubenstein 1995). Namely, what is involved are questions that, in their specific content, structure, or wording, lead respondents to give a certain answer (Moser and Kalton 1972). The two terms are generally used to describe the same type of questions (Cohen and Harnick 1980; Lyon 1999) or in some cases are understood such that one is a subcategory of the other, like with Gudjonsson (1984) who presents “leading questions” as the most subtle type of “suggestive questions”. The response to a leading question can be less accurate, distorted, or even untrue. The effects of suggestibility have been widely debated concerning police interrogations. Schooler and Loftus (1993) noted that the reporting errors of eyewitnesses arose from two types of suggestion implementation. First, delayed retrieval errors occur when misinformation or another form of suggestion is presented to the subject after a certain event and is incorporated into one’s memory before the questioning. Second, immediate misinformation acceptance refers to the suggestion implied to the subject during the interview in the particular form of the questions. The latter means that the questions themselves are biased and formulated in a way to affect a person’s memory and lead towards a desired answer (“leading questions”). Studies show that during questioning giving negative feedback and even repeating the same question can lead to possibly untrue testimonies since the subject might then adapt their answer to the perceived expectations of the interrogator (Gudjonsson 1984; Sharman and Danby 2022). It has been proven that leading questioning significantly affects the accuracy of witness testimony (Gous and Wheatcroft 2020). Moreover, the suggestions might contaminate a person’s memory and alter the way they recall a certain event (Loftus 2003; Loftus and Zanni 1975).

On the other hand, leading questions in social sciences research also influence the answers of respondents, altering the research results and thereby reducing their validity and reliability (Di Bella 2009). Research shows that respondents indeed tend to choose the answers suggested in the questions themselves more often than other answers, which importantly affects the quality of the data collected (Smit et al. 1997). In addition, since they can influence a person’s memory, a single leading question can distort the answers to other questions in the survey or interview (Di Bella 2009). It is important to note that the survey is not the only data collection method susceptible to suggestibility. It emerges as an issue within all methods involving the questioning of subjects and can have a strong impact in interviews as well. A suggestible interviewer's behaviour can easily lead the respondent to answer in a given way, even to provide untrue or dishonest answers (Smit et al. 1997).

The described shift in opinion away from a factual perspective presents an important measurement error in collected data. Authors generally agree that survey answers will always be collected with a certain degree of error, blurring the true estimates, effects, and relationships (Bound et al. 2001). In the case of leading questions, the error is not due to invalid or unreliable measurement, but measurement bias (Dutka and Frankel 1991). Leading questions are thus distinguished from other poorly worded questions primarily by the type of error they create, implying a specific—biased—response with their suggestibility. Strongly leading questions are often used intentionally, leading to acceptable or wanted answers, such as in marketing research to manipulatively present consumers’ liking of a product (Janatabadi and Ermagun 2022) or in push polls to erroneously present the support of a certain political candidate or political topic (Gherghina and Qvortrup 2024), and/or influence the voting outcome (Streb and Pinkus 2004). On the other hand, even nuances in question-wording, causing the greater prevalence of particular answers, not necessarily done intentionally, can also be understood in terms of leading questions (Freedman et al. 2003). These include the negative or positive wording of a question or interviewer attitudes that affect the course of the interview and objectivity (Houtkoop-Steenstra 1997), creating bias by leading the respondents to a given answer and in turn causing a measurement error. In this case, the answers will not reflect a true perspective (Robinson 2018). Due to their seemingly innocent suggestibility, those questions are relatively common in survey research as well.

For controversial topics, such as the one this article is based on (attitudes concerning minority groups), leading questions are often found in survey research. For measuring negative attitudes to migrants, for example, authors might use negative wording or even somewhat leading questions consisting of bias in structure (see Fawzi 2019; Hameleers et al. 2017). Biglin and Purdam (2023) note the lack of objectivity in measuring public attitudes regarding migrants and expose how some of the leading social surveys adopt suggestible approaches when questioning respondents. Given its controversy, the topic of immigration seems especially prone to this, while (incorrectly) measured negative public perceptions can be exploited to justify certain negative policy responses (Leigh 2017).

Considering the different types and levels of leading questions, we aim to simulate what happens to respondents’ opinions when subjected to different levels of question manipulation—from simple bias in wording, also seen in common survey research, to stronger suggestions as mostly used in the commercial or political spheres (such as push polls). Assessing the effects of different levels of suggestibility allows us to understand differences in their effects on measured opinion and their contribution to measurement errors. The research first aims to test whether the public is easily swayed by leading questions in surveys and to that end the share of respondents susceptible to suggestion is assessed. Second, it explores whether certain group types of respondents in terms of their susceptibility exist. Third, since some respondents tend to be more susceptible to suggestion than others, we investigate the relationship between several demographic and personality characteristics and susceptibility.

1.1 Types of leading questions

According to research, suggestibility can take a variety of forms. It can be embedded in the question in the way the question is formulated, by using certain manipulative phrases or words, or by the type of possible answers offered (Di Bella 2009). It can be embedded in the question in the way the question is formulated, by using certain manipulative phrases or words, or by the type of possible answers offered (Di Bella 2009). Different question structures were found to have a different impact on the extent to which they influenced the subjects. For example, children in the study by Powell and Roberts (2002) were more likely to follow the suggestion within a closed question offering two incorrect alternatives than within an open question that included false information. Closed questions are assumed to lead respondents to guess between the answers offered when they are unsure, while open questions offer greater freedom to say that they do not know the answer (Dent and Stephenson 1979). For adults, Sharman and Powell (2011) found that leading/desired answers were more often given when the question narrowed the response options and gave very specific details, and when the question was open but relied on the presumption of certain information.

Gudjonsson (1984) recognises three types of what he calls suggestive questions: 1. leading questions, containing a certain expectation of something being present without knowing if it actually was (such as: “Did the person’s glasses break?”, implying that this person was wearing glasses); 2. affirmative questions, expecting a positive response (for example, “Were they convicted 6 weeks after the arrest?”, without first confirming whether they had been convicted at all); and 3. false alternative questions, offering alternatives that are not necessarily true (“Does she have one or two children?”) (Gudjonsson 1984; Warren et al. 1991).

Sharman and Powell (2011) present four types of leading questions using the example of testifying about a robbery. All four question types falsely suggest that a shotgun had been involved in the incident: 1. closed questions, which offer a yes or no answer (“Did the robber use a shotgun?”); 2. closed presumptive questions, leading the respondent to simply accept that there was a shotgun (“He used a shotgun, didn’t he?”); 3. closed specific questions, adding very specific details about the shotgun, causing the respondent to think about the detail rather than considering whether the weapon was actually there or not (“Did he use a shotgun with a black barrel and a brown stock?”); and 4. open presumptive questions that presume information and encourage subjects to think about the details of the suggested information, even though it is not correct (“Tell me about the shotgun that the robber used to hold up the bank”).

A general rule applies in surveys that leading questions should be avoided at all costs (Krosnick 2017). In opinion surveys, the use of negatively or positively worded questions/items is expected. Choi and Pak (2005) demonstrated that such wording can also produce biased answers because positively worded questions lead to more positive answers and vice versa. This is also called acquiescence, reflected in respondents’ tendency to agree with a statement without considering the content of the item (Colosi and Bureau 2005; Krosnick 1999). Bias can also be present due to respondents’ desire to present themselves in a socially acceptable way. Such bias is called social desirability bias and is more evident when a research topic is linked to sociably desirable or undesirable behaviour or attitudes (Bradburn et al. 2004). Scott and Steward (2018) state that a leading question “is an interrogative expression containing a mechanism potentially capable of systematically biasing responses. Those mechanisms may operate via any one of, or combination of, three pathways: biasing structure, biasing content, and/or biasing context”. Regarding the biasing structure of a question, the authors show that the question itself can include an answer (“The product worked well, didn’t it?”), steer a respondent to answer by logic (“Studies show that…. Do you think…”), incorporate a presumed fact (“How fast was the car going when it went through the Give Way sign”), avoiding or excluding information (use of an unbalanced scale) or “yes/no” questions (they always include facts). The biasing of question content can be done by including unsupported assertions/hypotheticals (“Some people say… Would you…”), supported facts (“Recent surveys show… Would you…”), broadly held beliefs or loaded words (references to cultural touchstones—the unacceptability of questioning some conclusions in a given cultural setting). With respect to the question context mechanism, bias may arise when respondents are aware of the desired outcome (the preface hints at the desired results of the survey), aware of the sponsor (they might want to please the sponsor), using preceding questions to influence the answer to a subsequent question and the interviewer giving non-verbal cues.

1.2 Susceptibility to suggestion

Early researchers already observed that not every person is equally susceptible to suggestion. Some people are more prone to altering their responses under the influence of leading questions, while others are more likely to stick to their answers. This can depend on one’s intelligence, memory, neuroticism, social desirability (Gudjonsson 1983; Howard and Hong 2002), compliance (Gudjonsson 1989), self-esteem (Singh and Gudjonsson 1984), and negative feedback (Gudjonsson 1984). Respondents showing greater suggestibility are more likely to be less intelligent, have a weaker memory about a certain event, be more neurotic, tend to behave more socially desirably (Gudjonsson 1983), are more anxious (Wolfradt and Meyer 1998), score higher on acquiescence, that is, have a tendency to accept something and be submissive (Eisen et al. 2002), and have lower self-esteem (Singh and Gudjonsson 1984). Similarly, as noted by Gudjonsson (1983), being unsure and not knowing all the information about the topic might lead someone to answer more in line with leading questions (Di Bella 2009). Suggestibility was also found to have a stronger effect when more time had elapsed between the event and the questioning (Singh and Gudjonsson 1984), and among people who had experienced more negative life events (Drake et al. 2008).

Studies also found that people who show greater susceptibility to leading questions are those whose coping mechanisms are emotion-based compared to those with problem-based coping styles (Howard and Hong 2002), those with an external locus of control and are extraverted (Paddock et al. 2000), and those with higher scores for social desirability and self-monitoring (Milstone 1993). A Portuguese study confirmed a significant positive correlation between suggestibility and neuroticism. They also found that suggestible people are more likely to be conforming and dutiful, feel the need to be respectful and cooperate with the authorities, strongly attached and loyal, self-controlled, hide negative feelings, and lack spontaneity. Further, those who tend to create overly positive impressions of themselves, those who are less imaginative, creative and intuitive, are also more likely to fall under the influence of suggestion (Pires et al. 2013).

While discussing a person’s suggestibility to leading questions, parallels can be drawn with a more general susceptibility to social influence. Oyibo and Vassileva (2019) established that people with higher neuroticism, lower openness, and lower conscientiousness are more likely to be susceptible to social influence. Peer influence can also have a stronger effect on adolescents who are impulsive and feel they are placed lower down the social hierarchy (Stautz and Cooper 2014). ‘Persuasibility’ was also found to be more typical for people with low self-esteem and less typical for those showing symptoms of acute neurotic anxiety (Janis 1954).

Another important variable influencing susceptibility to suggestions is age, with children being more prone to following the lead of questions than adults. It was shown that the younger the child, the easier it is to mislead them, with kindergarten children more often changing their answers according to the lead (Cassel et al. 1996). Among adults, some studies show that older adults tend to accept more suggestions than younger adults (Coxon and Valentine 1997; Mitchell et al. 2003; Polczyk et al. 2004), and are also more confident in the false memories implanted into the suggestions presented to them (Mitchell et al. 2003).

An interesting phenomenon described by social psychologists is contra-suggestion (Bogardus 1920; Stern 2017). The suggestion could be translated into action differently. It can be immediate (a suggested idea is promptly acted upon), mediate (a suggested idea is acted upon after some time has passed and can be modified), or contra-suggestive (some people and children respond oppositely to what is suggested). The latter is more characteristic of individuals with a greater sense of individuality and a lower level of conformity. These individuals internalised social cooperation to a smaller extent. In children, contra-suggestive behaviour is described as the pretence of strength meant to hide consciousness of weakness (Stern 2017).

The abovementioned findings about demographic characteristics, personality traits, and suggestion are mainly based on research on witness interrogation techniques. Investigators use leading questions to confuse and expose a dishonest and unreliable witness (Gous and Wheatcroft 2020). To measure interrogative suggestibility, Gudjonsson constructed the suggestibility scale GSS1 (Gudjonsson 1984) and examined factors related to suggestibility in forensic groups (for example Gudjonsson 1991). Our research builds on these findings and explores the relationship between demographical and personality characteristics and suggestibility in the context of misleading public opinion surveys or push polls. Although the latter are frequently present in everyday lives, their effect on respondents is less addressed in scientific literature. In a related but more extensively studied topic, misleading marketing has significant impact on consumer behaviour. Unsupported and misleading claims about healthcare products or services can lead to erroneous decisions, adversely affecting people’s health (Aagaard 2020; Brank et al. 2006) Push-polls can be considered a form of false advertising, releasing hypothetical and false information about political candidates to influence public opinion rather than measure it (Gerstmann and Streb 2004). These polls are typically brief, aiming to reach a wide audience without collecting or analysing data. Respondents often fail to recognize false political advertising, leading to influenced opinions (Gerstmann and Streb 2004). Fredén (2021) demonstrated that push-polls could increase false memories of fabricated news stories. Individuals exposed to such stories within push-polls were more likely to recall these stories as true after some time had passed. This issue extends to scientific social science research, where misleading data can negatively impact policymaking, as illustrated by Wolf (2007). Urbatsch (2023) highlights the prevalence of poorly worded and misleading questions in general social surveys. Vague or context-lacking questions can yield different responses compared to more specific ones. For example, people may initially state they cannot imagine any acceptable circumstance for a police officer to hit a citizen (which is a socially desirable answer) but might concede to such behaviour under specific circumstances when context is provided (Brank et al. 2006; Urbatsch 2023).

In conclusion, misleading statements in surveys, polls, or advertisements are a critical area of study. Susceptibility to such statements plays a crucial role in the internalization of erroneous information. Therefore, understanding susceptibility and the associated factors is a primary objective of this research.

2 Methods

A survey was performed to measure attitudes towards minority groups in Slovenia. The data were collected in August 2023 and included 613 participants aged 18–65 years. The sampling frame included members of the JazVem online panel. Stratified random sampling was applied. Strata were defined according to gender, age and education. The response rate was 27%. The data were weighted according to gender, age, region and education so that the final demographic structure reflected the population structure in 2023. The information on the population demographic structure was obtained from the Statistical Office of the Republic of Slovenia. Opinions on minority groups were measured by questions that ranged from neutral to leading—the latter reflecting a more negative opinion on minority groups. The research was ethically approved by the Ethics Committee of the Faculty of Criminal Justice and Security (approval no. 1909–2023).

2.1 Sample description

The survey included 613 respondents, with 320 (52.2%) of them being male, 184 (30.1%) having at least a university education and 415 (67.7%) being employed, 33 (5.6%) unemployed, 68 (11.1%) retired, 64 (10.7%) students, 21 (3.5%) having some other occupational status (Table 1). The mean age (SD) of the participants was 43.3 (13.6) years. Less than 10% (43; 7.1%) stated that they were members of a minority group. The population characteristics were retrieved from the Statistical Office of the Republic of Slovenia (2024) but included citizens from 15 years of age onward (except for age, where citizens aged 18–65 were included in the calculation of the mean and standard deviation), meaning that some differences in the demographic structure of the sample, where the age span was 18–65 years, and the population may be expected.

Table 1 Sample and population demographic characteristics
Full size table

2.2 Questionnaire description

The questionnaire included questions about the perception of minority groups, social desirability, and the personality of the respondents. It was pilot-tested on 179 undergraduate students of the Faculty of Criminal Justice and Security. The students provided comments about the comprehensibility of the items and the structure of the survey.

The final aimed to measure three possible sources of bias: social desirability bias, negative wording bias, and leading questions bias.

Social desirability was measured by Hays et al.’s (1989) SDRS-5 scale. The items were translated to the Slovenian language by two independent translators and, after the two reached an agreement on the translation, the items were back-translated to English. The comparison of the original and back-translated scale showed there was no change in the meaning of the items. An exemplary item is “There have been occasions when I took advantage of someone”. All items were measured on a 5-point Likert agreement scale. Factor analysis showed a single factor structure, although two items did not have considerable weight on the factor. The final scale consisted of three items, the remaining two being “I sometimes try to get even rather than forgive and forget” and “I sometimes feel resentful when I don't get my way”. All three items were reverse-coded, and a composite variable was calculated as the average of the three items. Higher values indicated higher social desirability. Cronbach’s α of this scale was 0.69.

To measure negative wording and leading question bias, a 4 × 4 questionnaire design was used. There were four blocks of questions, with the blocks varying in the type of question wording (negative or positive) and suggestibility. Each block included four items/questions covering four different minority-related topics (culture, employment, crime, conflicts). The order of the items within each block was random.

In this way, we aimed to test the effects of different suggestibility levels rather than the mere existence of suggestibility (Smit et al. 1997) or different structure types of leading questions (such as Powell and Roberts 2002; Sharman and Powell 2011). To minimise the impact of pre-existing opinions on the topic, susceptibility to suggestion was measured as a shift from answers regarding neutral items to answers in the context of gradually stronger suggestions. We follow previous research manipulating the strength of suggestibility (Gous et al. 2020) with a more structured approach on four levels. To maintain the content consistency of questions, all levels included questions on the same four sub-topics.

The first block included positively worded or neutral items, where participants expressed their opinion on each of the topics on a five-point scale, from strongly negative to strongly positive. Higher values reflected more positive opinions to minority groups. An example item is “What is your opinion on the presence of minority group culture in Slovenian society?”.

The three remaining blocks included negatively worded items where higher values indicated more negative opinions about minority groups. The items were measured on a 5-point Likert scale of agreement, from strongly disagree to strongly agree. The three remaining blocks varied in the suggestibility of the instruction accompanying each block. The first block was non-leading. An example item about the opinion on minority culture was: “To what extent do you agree or disagree that minority groups should adapt to Slovenian customs as much as possible?”. The wording was negative, but the instruction did not lead respondents to answer in a particular way. Building on Scott and Stewart’s (2018) definition of types of leading questions bias, the second block of questions was somewhat leading by including bias in structure. The slight push towards the desired answer was achieved by asking respondents if they agreed with an item. An example of an item about the culture was: “Do you agree that minority groups contribute to intercultural conflicts?”. The third block of negatively worded yet leading questions was strongly leading. The suggestibility was achieved by incorporating content and context bias. The instruction included a presumed fact (content bias) and was placed at the end where the previous block had already hinted at the direction of the desired outcome (context bias). An example item is: “There is a lot of reporting on conflicts between minority groups and Slovenian residents. To what extent do you agree or disagree that minority groups undermine Slovenian culture?”.

The questionnaire structure is illustrated in Table 2.

Table 2 The structure of the part of the questionnaire measuring bias in negative wording and leading questions’ bias
Full size table

Exploratory factor analysis including the four blocks of items resulted in a two-factor structure (KMO = 0.92; Bartlett’s χ2 (120) = 6544.1; p < 0.001) with the items from the first, neutral, block loading considerably (weights > 0.40) on the first and items from the second till fourth block (non-leading, somewhat leading and strongly leading) on the second factor. Reliability as measured by Cronbach’s α for the first and second factors was 0.82 and 0.93, respectively. Composite variables for each block were calculated as the average of the responses provided for the four items within the block. Higher values indicated a more negative opinion about minority groups. To this end, the items in the first block were reverse-coded before computing a composite variable.

Questions on minority attitudes were followed by a set of questions on social desirability.

Personality was measured by BFI-10 (Rammstedt and John 2007), although additional items were used as previous research in Slovenia showed they performed better than using solely original items (Erčulj and Pavšič Mrevlje 2023). All items were measured on a 5-point Likert agreement scale. Factor analysis resulted in a 5-factor solution. Each personality trait was measured by two items. Neuroticism with the reverse-coded item “is relaxed, handles stress well” and “gets nervous easily” (α = 0.64), extraversion by “is sociable” and “is communicative” (α = 0.81), conscientiousness by “does a thorough job” and “is exact” (α = 0.76), and openness by “is resourceful” and “is original” (α = 0.81). Agreeableness was measured by two items “is compassionate” and “is generally trusting”, albeit only the latter had a substantial loading on the factor. This trait was not used in further statistical analysis.

2.3 Statistical analysis

Categorical variables were described by frequencies and percentages, continuous by means and standard deviations when normally distributed, and by medians and interquartile ranges (IQR) otherwise. Hierarchical clustering using Ward’s method of aggregation and squared Euclidian distance as the measure of similarity was applied to identify groups of respondents in terms of their suggestibility (provided answers to the composite variables with negative wording). Multiple multinominal logistic regression was employed to test the association between several demographic, social and personality characteristics and suggestibility. Being a member of a minority group was included in the model as a control variable. All of the statistical testing was done on a 5% significance level, using SPSS, v. 29.

3 Results

The descriptive statistics of the composite variable of the three sets of items with negative wording—non-leading, somewhat leading, and strongly leading—and a set with neutral items are summarised in Fig. 1. The median (IQR) of the answers to neutral items was 2.75 (2.25–3.25), negatively worded items that were non-leading 3.00 (2.75–3.50), negatively worded items that were somewhat misleading 3.00 (2.50–3.75) and to negatively worded items that were strongly leading 3.00 (2.50–3.50). The median is higher for the negatively worded items reflecting a more negative opinion on minority groups. Although the median remains unchanged for the three sets of negatively worded questions, the variability in answers increases, when the question is leading.

Fig. 1
figure 1

Distribution of answers by type of question

Full size image

Hierarchical clustering resulted in three groups of respondents according to their answers to the three sets of negatively worded items. Those in the first group (n = 118; 19%) were negatively oriented to the minority group and the leading question reinforced this negativity further by shifting their answers towards an even more negative opinion. The second group (n = 322; 53%) were respondents whose opinion shifted from neutral towards more negative when leading questions were asked. The third group (n = 173; 28%) were respondents whose opinion shifted towards more positive when leading questions were asked. Building on this classification and treating the shift in opinion as considerable when the value of the composite variable changed by at least one-half when a leading question had been asked, five groups of participants were identified (Fig. 2). Non-suggestible respondents (n = 218; 35% [95% CI: 32–40]) are those whose opinion did not change regardless of the suggestibility of the question asked. Strongly suggestible respondents (n = 92; 15% [95% CI: 12–18]) are those whose opinion shifted to more negative already after asking a somewhat leading question. Mildly suggestible respondents (n = 72; 12% [95% CI: 9–15]) were those whose opinion shifted considerably towards negative after asking a strongly leading question, but remained more or less unchanged after a somewhat leading question. Strongly contra-suggestible respondents (n = 150; 24% [95% CI: 21–28]) are those who shifted their opinion to less negative following somewhat leading questions. The final group of respondents are those who are mildly contra-suggestible (n = 82; 13% [95% CI: 11–16]). These respondents altered their opinion to more positive after facing strongly leading questions.

Fig. 2
figure 2

The five groups of respondents according to suggestibility

Full size image

For the purpose of examining the relationship between personality traits, social desirability of answers, demographic variables and susceptibility to suggestion, the respondents were divided into three basic groups: non-suggestible, suggestible and contra-suggestible. The latter two groups were compared to the non-suggestible group with respect to the above-mentioned characteristics. The results of multiple multinominal logistic regression are illustrated in Fig. 3. Younger people are more susceptible to suggestion—changing their opinion to a greater extent when faced with a leading question. The odds for susceptibility are about 2% lower with each additional year of age (OR [95% CI]: 0.98 [0.96–0.99]) when controlling for other independent variables in the regression model. Male respondents have about 3% [OR [95% CI]: 1.03 [0.67–1.58] higher odds of suggestibility and about 26% [OR [95% CI]: 1.26 [0.85–1.85] higher odds of contra-suggestibility than women. Lower education is associated with higher suggestibility [OR [95% CI]: 1.12 [0.71–1.78] and lower contra-suggestibility [OR [95% CI]: 0.83 [0.55–1.25. Members of a minority group are less suggestible [OR [95% CI]: 0.54 [0.23–1.24] or contra-suggestible [OR [95% CI]: 0.69 [0.34–1.41] than non-minority members. The conclusions regarding gender, education and minority groups cannot be generalised to the whole population. Minority group affiliation was only included as a control variable as it could influence other investigated associations, noting the topic addressed in the survey.

Fig. 3
figure 3

Odds ratios (95% CI) for each characteristic, comparing suggestible (left-hand side) and contra—suggestible (right-hand side) respondents with non-suggestible respondents

Full size image

Among the personality traits, openness and conscientiousness are associated with suggestibility, but not with contra-suggestibility. People who are more open to new experience are less suggestible (OR [95% CI]: 0.67 [0.46–0.97]). Individuals who are more conscientious are more susceptible to suggestion (OR [95% CI]: 1.53 [1.03–2.28]). Higher neuroticism and extraversion in the sample were associated with greater suggestibility, although the association was not statistically significant. Respondents with each additional score on neuroticism have about 28% higher odds of susceptibility (OR [95% CI]: 1.28 [0.97–1.96]). More extraverted respondents are more susceptible to suggestion (OR [95% CI]: 1.30 [0.97–1.76]). Social desirability is not associated with suggestibility but is associated with contra-suggestibility. Those who are more inclined to provide socially desirable answers have higher odds of being contra-susceptible to suggestion (OR [95% CI]: 1.36 [1.06–1.75]). Regarding the remaining variables, although not statistically significant, the calculated odds ratios show lower contra-suggestibility for minorities, less educated, less open and less extravert respondents.

4 Discussion

In the social sciences, the measurement error (the difference between the obtained answer and the genuine one) tends to be larger than in the natural sciences due to the softer approach to measuring a given researched phenomenon. Data collection often includes the use of a questionnaire. Although there are many instructions on how to prepare a»good« questionnaire that is valid and reliable, many factors can contribute to a measurement error. Several researchers have demonstrated that various errors can be made simply by using closed or open questions, providing a list of possible answers, using unbalanced or balanced scales with a different number of response options or scale points, including a»don't know« category, positive or negative question formulation etc. (Brace 2008; Choi and Pak 2005; Gueorguieva et al. 2020; Weng 2004). Several of the mentioned aspects closely intertwine with question suggestibility. Scott and Steward (2018) mention unbalanced scales as one form of a leading question biasing its structure by avoiding or omitting information. Choi and Pak (2005) argue that a positive or negative form of a question can also lead to bias in an answer as respondents tend to answer more negatively to negatively worded items and more positively to a positively worded item. Positively or negatively worded items are frequently used in research that measures attitudes—for example, public opinion surveys. Choi and Pak's claim was supported by the findings of our research as factor analysis performed on items measuring attitudes to minority groups resulted in a two-factor solution. The block of neutral items loaded on a separate factor from the three remaining blocks of content-wise similar, yet negatively worded items. When comparing the answers of respondents to neutral against negatively worded items, the median response value was indeed lower than the negatively worded items. The lower median value indicated a more favourable opinion on minority groups. The use of negatively worded items therefore depicts less favourable public opinion concerning minority groups. Negatively worded items were further divided according to their suggestibility into non-leading, somewhat and strongly leading. Although the median value for all three was the same, the variability of responses increased when an item was leading in terms of its structure, content and/or context. We named the block of items including bias in structure “somewhat leading” as the instruction omitted the possibility of disagreeing with the content of the item (»To what extent do you agree…«) and the block of items including content and context bias as “strongly leading” since the content included information on media reporting on conflicts between minority groups and other residents (a presumed fact) and was placed after the block of items already hinting at the desired answer (context bias). The greater variability of answers to these two blocks of items in comparison to the non-leading (albeit negatively worded) one suggests that the respondents’ answers were less homogenous and their opinion more prone to changing after a leading question was asked. Indeed, the clustering of respondents by their answers to the three blocks of negatively worded items (more specifically their composite values) led to three groups of respondents: non-suggestible (their answer did not change considerably when asked a leading question), suggestible (their answer changed in the desired direction when asked a leading question) and contra-suggestible (their answer changed in the undesired/opposite direction when asked a leading question). The latter two groups can be further divided into mildly and strongly suggestible or contra-suggestible based on how leading the question must be to provoke a considerable change in the answer. The non-suggestible group consisted of 35% of respondents, the suggestible group of 27% and the contra-suggestible one of 38% of respondents. One objective of the research was to assess the share of suggestible research participants. The 95% confidence interval indicates that this share is between 23 and 30%. The share of contra-suggestible respondents outweighed the share of suggestible respondents. Despite the thorough literature search on the topic of contra-suggestibility, not many results emerged. As suggested by social psychologists (Bogardus 1920; Stern 2017), contra-suggestibility could be one form by which the suggestion is translated into action (= answer). Adults behaving in a contra-suggestible way tend to be more individualistic and conform less to norms. This claim contradicts the results of our research where contra-suggestibility, after controlling for other demographic and personal characteristics of an individual, was statistically significantly associated only with social desirability. Our results show that contra-suggestible individuals strive towards socially desirable answers to a greater extent than non-suggestible individuals do. This finding must be placed and explained in the context of the research topic of examining attitudes to minority groups. Acting contra-suggestibly when the question leads to a less favourable opinion on minority groups is in fact more socially desirable and at the same time illustrates stronger conformity to the expected or ‘correct’ behaviour—accepting people of a different race, culture and values. Although respondents did not change their opinion in the direction desired by the researcher, they changed the opinion in the publicly desired direction. Contra-suggestibility can accordingly be understood as susceptibility to the suggestion as well. The change in opinion, irrespective of in which direction, is a direct consequence of a leading question. In the light of this argument, the share of respondents susceptible to suggestion increases to 65% [95% CI: 61–68] because it includes both suggestible and contra-suggestible individuals. In this sense, the finding is in line with Gudjonsson (1989) or Milstone (1993) who argue that people behaving in a more socially desirable manner tend to be more susceptible to (in our case contra-) suggestion.

Considering only the group of suggestible respondents—those who changed their opinion in direction desired by the researcher—the factors associated with suggestibility in our research are age and personality traits. The older the respondents are, the less susceptible to suggestion they tend to be. Some studies demonstrated that children (for example, Cassel et al. 1996) and the elderly (Coxon and Valentine 1997; Mitchell et al. 2003; Polczyk et al. 2004) are more prone to following the suggestion than non-elderly adults. Our research included only adults, with the oldest adults being 65 years old. Mitchell et al. (2003) based their findings on comparing elderly participants with younger adults, where the mean age of the elderly participants was 74 years. The participants in our research were younger. Social research shows (Visser and Krosnick 1998) that the personal importance assigned to attitudes on policy issues becomes stronger towards middle age and is lower among the youngest and as aging continues. In this regard, our finding of lower suggestibility toward the researched topic with increasing years is expected. Although fewer more educated respondents and members of minority groups in the sample changed their answer in the desired direction after being faced with suggestive question, the association was not statistically significant when controlling for all variables included in the regression model. For both variables, the confidence interval for the odds ratio was quite wide due to the smaller number of minority group members and highly educated participants within the suggestible group of participants. As regards personality traits, our research showed that conscientiousness and openness are associated with susceptibility to suggestion. Openness is a personality trait including creativeness, flexibility, curiousness and unconventionality (McCrae 1996). People who are more creative, flexible, curious and unconventional tend to be less susceptible to suggestion (Oyibo and Vassileva 2019), as also supported by the results of our study. Contrary to openness, conscientiousness was found to be positively associated with suggestibility. This is not in line with Oyibo and Vassileva’s (2019) findings which indicate that susceptibility to suggestion is higher in less conscientious people. Conscientious individuals are orderly, dependable and achievement-driven (Barrick et al. 1993). Conscientiousness helps people who are more open (individualistic, unconventional) to express their non-normative opinions (Packer 2010). In our case, however, it would be expected that a more open person would hold a more favourable opinion on minority groups. That is why open respondents more likely ‘stood their ground’ and did not alter their opinion to a more negative one following the leading question. They resisted acting in a non-normative way, although still acted individualistically—being true to their beliefs. Nonetheless, conscientiousness still helped people to express their non-normative behaviour by taking the lead in the question and changing their opinion on minority groups to a more negative one. Our results indicated that more neurotic and more extravert respondents in the sample had higher odds of susceptibility—they were more likely to change their answer in the desired direction, albeit the association was not statistically significant thereby not confirming the studies by Gudjonsson (1983), Pires et al. (2013) and Paddock et al. (2000).

To summarise our findings, the estimated share of respondents susceptible to suggestion in intentionally or unintentionally poorly designed or misleading surveys ranges from 61 to 68%. When addressing sensitive and controversial topics possibly provoking a strong emotional response and strong social desirability tendencies in respondents, suggestibility might be expressed in two possible ways—responding in a suggestible or a contra-suggestible manner. In such cases, contra-suggestibility is associated with the strong social desirability of answers and suggestibility with the age and personality traits of respondents. The designers of push polls and misleading surveys should be aware of this phenomenon. Although trying to provoke a desired answer in respondents by using strongly suggestible questions, they might end up with just the opposite effect. On the other hand, the bias produced by public opinion surveys using mildly suggestive questions could be counterbalanced by the answers given by contra-suggestible respondents.

The implications of our research are that measurement error in surveys is increased by the negative wording of a question as well as by the use of structurally or content- and context-wise leading questions. The error further increases when research includes younger and conscientious respondents or those less open to new experiences. Social desirability plays an important role when suggestibility is present with respect to a sensitive and controversial topic. The use of suggestible question in push polls and intentionally misleading surveys might backfire in the presence of respondents’ contra-suggestibility to the extent that it levels out the measurement error referring to more gullible and susceptible respondents.

There are some limitations of this study. The survey content was specific and the generalisability of the results to all, less controversial, survey topics is questionable. Although the wording of items was different, the content was similar, which may have influenced the responses of the study participants. The random order of items hopefully improved this drawback, albeit it would be better to use an experimental design. Such a design would, however, influence the sample size and/or cost aspects of the study. In addition, specific to our topic, the propensity for social desirability might be somewhat associated with one’s ideological opinion on migrants or the acceptance of them. However, the collected data did not include political attitudes as a test variable. They would only be relevant for both political poles, and based on the last Slovenian Public Opinion survey (2024) political affiliation is somewhat balanced, with less than 10% of respondents positioning themselves at each pole. Still, it would be expected the ideological attitudes would be reflected in the answers of respondents from the beginning and not only after they were faced with leading question. Repetition of the study and different survey content would lead to more rigorous conclusions.