Authoritarian governments have many ways to control the media. In addition to censorship, they can force media outlets to broadcast nothing but praise of political leaders or news that glorifies the regime. However, such official propaganda, or in other words, “hard propaganda,” is not an effective tool to win people’s hearts and minds because its one-sidedness may look excessive and unreal, thereby weakening its appeal (Huang, 2018). For this reason, regimes also utilize “soft propaganda” to guide public opinions (Huang, 2015, p. 419). Instead of completely censoring unfavorable information, these governments allow the coexistence of both critical and pro-regime voices in the online sphere, in which citizens can view or even discuss some controversial issues (Egorov et al., 2009; Qin et al., 2017). The authorities then neutralize regime critics by recruiting undercover commentators (Gunitsky, 2015; King et al., 2017), mobilizing regime supporters to leave comments (Han, 2015a, 2015b), and amplifying the voices of the patriots (Fang & Repnikova, 2018). This approach is potentially more influential because, compared to one-sided propaganda, lively online discussions look more authentic and trustworthy.

China, as one of the states that have tight control of the Internet, has actively adopted soft propaganda to guide public opinions. Recent studies provide suggestive evidence for its success. For instance, some observe a rising trend of nationalism on the Chinese Internet (Hyun et al., 2014; Jiang, 2012). Others point out the proliferation of “little pinks” and “volunteered 50-cent members” (ziganwu)—netizens who volunteer themselves to defend the regime (Fang & Repnikova, 2018; King et al., 2017). There are also “wolf warriors,” netizens who manage to bypass the “Great Firewall,” so that they can visit banned foreign social media and rebut foreigners who have issues with China (Teixeira, 2020).

While these examples seem to attest to the Chinese government’s success in shaping public opinions, empirical evidence remains woefully inadequate. In this article, we try to fill this gap by exploring two questions: (1) to what extent do these pro-regime comments appeal to ordinary Chinese netizens?; and (2) would the presence of pro-regime discourses make citizens more supportive of the regime?

To answer the first question, we use an online survey to track Chinese social media users’ self-selected comment exposure over time. To answer the second question, we make use of an online experiment to examine if netizens’ policy support will change in the absence of online comment exposure. Online comments in this paper are comments taken from the comment section of Sina Weibo,Footnote 1 which is one of the most popular and most regulated social media platforms in China (Vuori & Paltemaa, 2015).

The empirical results show that when given an option, the survey respondents tend to self-select to expose themselves to critical comments rather than pro-regime comments over time. We also find that compared to those who do not have a chance to view comments, those who can view comments—either critical or pro-regime—are significantly less supportive of the underlying policies.

The findings advance our understanding of online propaganda and information control in authoritarian regimes. We argue that although the Chinese authorities have gone to great lengths to shape public opinion through cajoling and coercion, ordinary Chinese may not passively and mechanically accept soft propaganda, especially when they can access critical comments. Nor do they seem to be mindless information recipients who would self-select to expose themselves to state-sponsored information. The highly regulated media environment seems unable to quell their demand for alternative information. In addition, the presence of pro-regime comments seems unable to neutralize the effects of critical comments by increasing public support of the regime. This could be why the Chinese government does not relinquish the use of hard censorship to control the media, despite the seeming proliferation of soft propaganda.

Literature Review

Social media play a vital role in political communication. Many authoritarian regimes impose tight controls over the Internet for this reason (Pan & Siegel, 2020). Although some scholars recognize social media as a liberalization tool that facilitates democratization (Diamond, 2010; Lynch, 2011), a growing body of literature argues that non-democracies are able to employ social media to reinforce state ideology and strengthen authoritarianism through information controls and regulations (Creemers, 2017).

In China, there are two recognized approaches to manipulating online information (Chen & Xu, 2017). The first approach is online censorship, which refers to the practice of online content deletion that aims to prevent netizens from accessing information deemed unfavorable to the regime (King et al., 2013). The second approach, which is less conspicuous, is to co-opt social media as an instrument to reinforce regime legitimacy (Gunitsky, 2015). Studies find that the Chinese government has been recruiting commentators to post online comments that promote the interest of the party-state (Han, 2015a, 2015b). Those comments, also known as 50-cent comments or shuijun comments, which seem to be authentic voices from ordinary netizens, are carefully crafted by the authorities for propaganda purposes (Bremmer, 2010; Deibert & Rohozinski, 2010; King et al., 2017; Miller, 2016). Such “soft propaganda” is expected to subtly influence public opinions and persuade citizens to believe in the state ideology (Greitens, 2013; Huang, 2015; Hung, 2010).

In addition to the above, the state also actively incorporates regime supporters’ discourses into its propaganda campaigns. Recent studies show the emergence of “little pink,” a term describing ordinary citizens who like to express their fervent support of the regime (Fang & Repnikova, 2018). By amplifying their voices and filtering out the voices of dissent, the state can turn patriotic netizens into “thought work collaborators” (Hung, 2010; Repnikova & Fang, 2018, p. 763).

Existing studies primarily focus on identifying pro-regime comments generated through astroturfing (King et al., 2017; Miller, 2016); seldom do they study the effects of fabricated and filtered comments on netizens. In this study, we focus on two outcome variables of interest: (1) individuals’ selective exposure to different online comments and (2) the effects of online discussions on individuals’ political attitudes toward regime legitimacy.

Theories and Hypotheses

Online Comments as an Echo Chamber

A surge in nationalism, together with tight ideology controls, has polarized China’s cyberspace, where clashes between those who firmly support the regime and those who dare to dissent from the government are commonplace (Chase & Mulvenon, 2002; Pan & Xu, 2018; Wu, 2013; Yang, 2014; Yang & Tang, 2018; Zhao, 2016). The growing nationalistic sentiment is partially due to a prolonged nationwide patriotic education (Hyun & Kim, 2015; Kennedy, 2009; Zhao, 1998). Fervent supporters of the party-state ideology are ubiquitous in China’s cyberspace and their voices are particularly noticeable when international tensions exist between China and other countries like Japan and the United States (Jiang, 2012; Qiu, 2006). Zhou (2005) shows that even well-educated and informed individuals in China may hold strong anti-Western and nationalistic values.

The echo chamber effects theory suggests that people favor information conforming with their pre-existing beliefs and tend to avoid ideologically uncongenial perspectives (Colleoni et al., 2014). The basic premise of the theory is that people’s cognitive resources are limited. When consuming information in a high-choice media environment, individuals do not have much time and patience to browse arguments from all sides, so they simply seek sources that reinforce, instead of challenging, their opinions (Arceneaux et al., 2012; Garrett, 2009). If this applies to Chinese social media, we shall expect to observe a selective consumption of information among Chinese social media users: regime supporters prefer information that defends the party-state, while others like to view more critical messages. Thus, we hypothesize that:

Hypothesis 1:

Given the constraints of time or cognitive resources, individuals will self-select to expose themselves to online information with similar ideological contents.

Online Comments as an Alternative Source of Information

Confirming one’s belief is possibly not the only reason why people consume online comments. From an evolutionary perspective, human beings have developed sophisticated sensory organs to process external information to enhance survival chances. Excluding information that is at odds with our beliefs is not conducive to self-preservation. On the contrary, exposure to diverse information helps us assess external risks and formulate appropriate responses.

In the context of authoritarian regimes, history is replete with examples where the authorities were unreliable information providers, especially in times of crises. After the explosion of a nuclear reactor at Chernobyl in 1986, the initial reaction of the Soviet authorities was to cover up the event. Workers at the nuclear plant were prohibited from sharing the news about the explosion, while telephone communications with the affected city were suspended. Similarly, during the early phase of the COVID-19 outbreak, the public security bureau arrested Li Wenliang, the whistle-blowing Wuhan-based medical doctor, on the charge of making false comments and spreading rumors (Ludwig, 2020; Zhang, 2020).

To avoid the risk of relying on official information, citizens in authoritarian regimes learn to share and obtain unofficial information (Nip & Fu, 2016; Tong & Lei, 2013). Zhu et al. (2013) show that grapevine news in China can significantly change popular perceptions. Chen and Yang (2019) show that once Chinese people are incentivized to access uncensored information, their demand for such information will increase. Hobbs and Roberts (2018) also provide empirical evidence that sudden blocking of information improves people’s motivation to search for blocked contents. These findings provide support to Geddes’s and Zaller’s (1989) argument that many citizens in authoritarian regimes are able to decipher reality from propaganda.

The demand for unofficial information may affect the consumption of online comments in authoritarian regimes. Pro-regime comments tend to echo the official discourse. As such, they provide little new information to recipients who look for diverse information. On the contrary, user-generated contents that deviate from the official discourse may offer comment viewers informed perspectives into an issue at hand that would otherwise be unavailable from the official media. For this reason, we posit our second hypothesis:

Hypothesis 2:

Given the constraints of time or cognitive resources, individuals will self-select to expose themselves to online information that deviates from the official discourse.

Effects of Online Comments on Regime Legitimacy

The reason why the authorities permit somewhat critical contents to coexist with regime-supportive contents in China’s cyberspace is the belief that they can guide public opinions or even drum up regime support with “soft propaganda.” In practice, this may not necessarily be the case. Studies show that online deliberations help expose individuals to different arguments and media framings. As such, individuals can have an opportunity to learn competing perspectives or ideas of which they were previously unaware. In other words, exposure to alternative information has the potential to stimulate critical thinking, thereby influencing individuals’ political attitudes (Tang & Huhe, 2014).

This exposure effect is also relevant to the Chinese context. Western-style democracy is absent in China. Chinese citizens have no direct way to select their political leaders beyond the village level. Accepting the political status quo, rather than challenging it, seems to be a sensible choice for them. Note, however, that the acceptance is not solely driven by scruples about the Leviathan. Historically, Chinese people tended to view the government as the head of a big family, whose decisions should be supported and followed (Shi, 2001). Rapid economic development in the reform era has further strengthened the political legitimacy of the ruling party. Empirical works also show that Chinese people have a high degree of trust in the central government even under the authoritarian control of the party-state (Chen, 2017; Li, 2016; Wang, 2005; Yang & Tang, 2010). Li (2004) observes that Chinese citizens generally believe that official policies, especially those implemented by the central government, bear beneficent intent. In other words, without arousing their awareness of diverse viewpoints, Chinese citizens in all likelihood are favorably disposed to the regime. Hence, we posit that:

Hypothesis 3:

Compared with individuals who lack access to online comments, individuals who are granted such access tend to show less policy support.

Note that Hypothesis 3 is related to policy support rather than regime support. The reason is that measuring regime support is problematic. In addition to official prohibition, it is also unlikely that survey respondents give truthful responses for fear that their discontent with the regime will cause them trouble (Krumpal, 2013). For this reason, we need an indirect measure to address this issue (Fisher, 1993). One way to examine regime support is to measure citizens’ support of the state’s proposed policies. In China, where the ruling party dictates the policy-making process, policies generally reflect the interests of the party-state. As a result, policy support may be viewed as an indicator of regime support (Chen et al., 1997; Lü, 2014; Tang & Huhe, 2014).

Research Design

Comments on Sina Weibo

Although outright disparagement of the regime or national leaders is nowhere seen in China’s cyberspace, political satires, innuendoes, and pungent comments on policies are ubiquitous. On many social media sites, one can see the co-existence of pro-regime comments and comments that are more critical. While in theory netizens can expose themselves to both types of comments, the government has made efforts to highlight regime-supportive comments in the comment section as a way to guide public opinions (King et al., 2017). Weibo is a case in point. If a post involves topics about the government and government officials, social policies, or political events, the comment section of that post is likely to show “recommended comments by the account holder.” These recommended comments are predominantly pro-regime. This is a way for the state to endorse and promote regime-supportive discourses from the grassroots (Repnikova & Fang, 2018).

Figure 1 displays an example of a comment section of a post (for contents of the post, see Fig. 3), which shows a local government’s decision to establish a rigorous disease reporting mechanism after the outbreak of coronavirus in Wuhan as a way to prevent any cover-up of the disease. The Hubei government has been accused of hiding the truth about the outbreak at an early stage, and this post reports the government’s response in late January. As shown in Fig. 1, those comments are rather supportive of the government. For instance, the third comment from the top praises the government for its confidence and transparency, while the fourth commentator emphasizes his or her trust in the state. The last two comments are cheerleading slogans.

Fig. 1
figure 1

Comment section of the Sina Weibo post

The users of Weibo can still find critical netizens’ comments, but in a different section. Similar to the retweet button on Twitter, the repost button of Weibo allows people to share a post on their personal Weibo page. They may also comment on the shared post. Weibo users can find out what people who share a post would say about it by clicking the “repost” (zhuanfa) button. Figure 2 provides an example. As is seen, “reposted comments” are decidedly more critical and satirical than the comment section (pinglun) displayed in Fig. 1. Most of them mock the government’s slow responses to the outbreak and question the government’s inaction at an early stage (Fig. 3).

Fig. 2
figure 2

Repost section of the Sina Weibo post

Fig. 3
figure 3

Sina Weibo post about establishing a disease reporting mechanism in Hubei. Notes: This is a sample news post shared on Weibo. This post is related to a local government’s decision to establish a rigorous disease reporting mechanism after the outbreak of coronavirus in Wuhan in early 2020 as a way to prevent any cover-up of the disease

In other words, the design of Weibo provides netizens a choice to view different types of comments. If they click the comment section, they expose themselves to those recommended pro-regime comments, be they produced by astroturfers or by the “little pinks”; if they click the repost section, they will see more critical and satirical opinions expressed by other netizens.

Online Survey Designs

We conducted two separate online surveys in November 2018 (Study 1) and February 2021 (Study 2), respectively. Online surveys implemented with online panels or outsourcing platforms have become increasingly popular in social science research (Clifford et al., 2015). While the generalizability of the findings based on online samples is a potential concern, Li et al. (2017) show that online samples in China provide a useful tool for research that makes inferences about the country’s netizens. Because we are primarily interested in studying Chinese netizens, whether our online samples are representative of the overall Chinese population is less of a concern here. That said, we took steps to maximize the generalizability of our findings. In addition to using quota sampling to recruit respondents representative of the Chinese population, we conducted two separate surveys to ensure that the findings are robust to a particular online sample recruited from a particular online platform. We discuss the details of each study in what follows.

Study 1

We issued the survey to 523 respondents through a well-established survey platform ( in mainland China targeting Chinese Internet users (i.e., Weibo). The sampling quota is based on the demographic distribution recorded in the 2010 population census of China based on five factors: gender, age, educational level, income and province. To avoid repeated responses, we only accepted responses with a unique IP address and with a completion time of no less than one minute (Goodman et al., 2013).

We model our survey upon the comment choices available to Weibo users. All respondents were given five actual Weibo posts regarding five somewhat controversial social policies (For exact wording, see Online Appendix 1-Study 1). In addition to the news script itself, they were provided with an opportunity to view one type of comment related to that piece of news. In particular, we provide the respondents with three options: (1) recommended comments (精选评论); (2) netizens’ comments (网友评论); and (3) skip comments (忽略). The first two options are analogous to Weibo’s comment and repost sections as shown in Figs. 1 and 2. We use the term “recommended comments” to make this option reminiscent of the respondents’ Weibo experience. This term, together with its political connotations, should be familiar to Chinese social media users, as it can be found on various social media platforms (i.e., Weibo and Wechat). In order to hint at unorthodox voices, we use the term “netizens’ comments.” To ensure that our respondents can detect the nuance between recommended comments and netizens’ comments, we include in the first policy news (i.e., the example news) one sample comment next to the recommended and netizens’ comments buttons, respectively (see Fig. 4). Respondents will not see sample comments again in the remaining policy news.

Fig. 4
figure 4

The example policy news. Notes: All respondents read this example policy news in Study 1 before viewing other policy news. As shown in this figure, we added a sample comment after the recommended comments and netizens’ comments buttons to give them an idea of the type of comments (i.e., pro-government or critical) they expect to see. In subsequent pieces of policy news, we did not provide any sample comment. This policy news is about a new rule concerning the purchase of multiple train tickets

If a respondent clicks the recommended comments button, they will see five pro-government or pro-policy comments. These comments either praised the policy or highlighted its importance. If a respondent clicks the netizens’ comments button, they will see five critical comments that question the feasibility and effectiveness of the policy. These comments were curated from actual comments, with some minor editing, related to the posts (for exact wordings, see Online Appendix 1).

In Weibo, users can choose to ignore comments about a post by clicking neither the comment nor the repost button. For this reason, we also provide the skip comments option to our survey respondents. The reason for allowing the respondents to view only one type of comment at a time is that online information seldom comes effortlessly and without cost. As discussed, people face time constraints and have limited cognitive resources. It is impossible for people to browse all kinds of information online. As such, they have to constantly decide the kind of information to which they want to expose themselves. We force them to choose in order to learn their preference for information.

Study 2

The basic design of Study 2 is similar to Study 1. We conducted Study 2 for two reasons. The first is to minimize potential house-effects of survey companies: we commissioned a different survey company to ensure that the results are not peculiar to the online panel of a specific company. The second reason is that Study 1 is not robust to some known potential biases such as question order effects. We designed Study 2 to minimize these potential biases. We detail the improvements below.

Unlike Study 1, we randomized all policy news in Study 2 to ensure that the results are not driven by a particular sequence of news being shown to respondents (for exact wordings of all policies, see Online Appendix 1). In addition, we also randomly assigned our respondents into two groups with different order of comment choice options. They saw either recommended comments placed on top of netizens’ comments and skip comments or netizens’ comments placed on top of recommended comments and skip comments. Done this way, we were able to minimize the order-effects of comment choice options. We also finetuned the wording of the options. For “recommended comments,” we rephrased this option to “comments recommended by the account holder” (博主精选评论), while we changed “netizens comments” to “comments reposted by netizens” (网友转发评论). By adding “account holder” and “reposted” to these options, we are able to bring these choices closer to the actual Weibo setting.Footnote 2

To further minimize the potential bias pertaining to a particular news example, we prepared three pieces of example news and drew only one random piece at time t = 1, so that not all respondents were exposed to the same example news. This step helps further reduce any idiosyncratic impact of the example news.

Last but not least, we added eight questions related to political attitudes in Study 2. We will use these questions, which come from Huang (2015) and Pan and Xu (2018), to control for the potential confounding effects of political attitudes on policy support. Table 1 contrasts the research design of Study 1 and Study 2.Footnote 3

Table 1 Comment choices over time: differences in research design between study 1 and study 2

Estimations and Results

Estimations of Comment Choice Over Time

Our variable of interest is \(N_{t}\), the sequence of their exposure to five (Study 1) or six (Study 2) pieces of policy news. We control for the respondents’ initial comment choice at time t = 1, as their initial choice is likely correlated with their attitude toward the government.

Our outcome of interest is respondents’ comment choice. In particular, each respondent read five (for study 1) or six policies (for study 2). After reading each policy news, they were allowed to choose between recommended comments, netizens’ comments, or skip comments. Because the outcome is categorical, we analyze the data using a multinomial logistic regression model (Gelman & Hill, 2006). When using multinomial logistic regressions, researchers need to specify an outcome as a baseline category for comparison. In our case, we choose netizens’ comments as the baseline group because we are primarily interested in finding one’s preference for pro-regime comments conditional on one’s decision to view a comment.Footnote 4

Given that we have three outcomes, the multinomial logistic regression requires us to run two multinomial logit equations, contrasting the baseline outcome with each alternative outcome. Because we are primarily interested in finding whether recommended or netizens’ comments appeal to the respondents, the multinomial logit equation of interest is as followsFootnote 5:

$$\Pr \left( {Y_{it} = \frac{{{\text{Recommended}}\;{\text{comments}}}}{{{\text{Netizens}}'\;{\text{comments}}}}} \right) = e^{{\alpha_{i} } + \beta N_{t} + \gamma_{1} A_{i} + \gamma_{2} B_{i} + X^{\prime}_{i} \omega + \varepsilon_{it}}$$

where \(Y_{it}\) denotes the probability of individual i choosing recommended comments at time t over choosing the baseline category netizens’ comments,\(\alpha_{i}\) is individual i’s intercept, \(\beta\) is the coefficient on the sequence of policy news \(N_{t}\) = 1, 2, …, 5, (6), \(\gamma_{1}\) and \(\gamma_{2}\) are coefficients on respondent i’s initial comment choice of recommended comments \(A_{i}\) and netizens’ comments \(B_{i}\),Footnote 6\(\omega\) is a vector of coefficients on a list of covariates \(X_{i } { }\) and \({ }\varepsilon_{it}\) is the error term clustered at the individual level and assumed to be independent and identically distributed.

The multinomial logistic regressions compare the choice of recommended comments vis-à-vis netizens’ comments. It would be instructive to pit recommended comments against non-recommended comments (i.e., netizens’ comments and skip comments). For this reason, we also converted the outcome variables into a binary choice (recommended v. non-recommended) and re-ran the specifications using random-effects logistic regressions.

Table 2 presents the regression results of different regression specifications. For multinomial logistic regressions, we show only the results of choosing recommended comments over the baseline group, netizens’ comments.Footnote 7 After controlling for respondents’ initial choices of recommended comments and netizens’ comments, the coefficients on time variable \(N_{t}\) are negative across all specifications. The results are consistent in Study 1, Study 2, and Studies 1 and 2 combined. We also find similar results in both the multinomial logistic model and the random-effects logistic model. In Specifications 3 and 4, we control for eight measures of individuals’ political attitudes, including their political efficacy, their resistance against the West. The coefficients on \(N_{t}\) remain the same. Thus, the results suggest that when considering respondents’ comment choice variation over time, the probability of respondents choosing recommended comments over netizens’ comments declines significantly. Rather, they are more likely to choose netizens’ comments as they continue to view more policies.

Table 2 Respondents’ choice of comments over time

The results suggest that those who initially chose to view pro-regime comments would not expose themselves repeatedly to pro-regime comments. In other words, we find no evidence that Chinese netizens’ comment-viewing habits are bifurcated, such that pro-regime (critical) netizens would consume exclusively pro-regime (critical) comments. Rather, as our respondents would self-select to expose themselves to critical netizens’ comments over time, despite the fact that their initial choice was pro-regime comments, the results support the hypothesis that Chinese netizens regard online comments as a source of alternative information (Hypothesis 2).

To visually depict respondents’ comment choice variation through the exposure to five pieces of policy news, we apply the multi-state survival model to generate a diagram. The model not only allows us to take respondents’ initial comment choice into account but also considers respondents’ comment choice at each time point and how it may affect their later choice. This model is based on the Markov assumption that the occurrence of event at time t = 2 depends only on the previous event at time t = 1 (Crowther & Lambert, 2017). The model is often used to examine the change of patients’ conditions under different medical states, as patients’ health condition can switch from the starting state (i.e., post-surgery state) to a transition state (i.e., relapse state) or even directly to the absorbing state (i.e., death) (Crowther & Lambert, 2017). Using the three-state survival model proposed by Crowther and Lambert (2017), we examine respondents’ comment choice switching tendency from recommended comments to netizens’ comments (see Fig. 5). We consider the choice of skip comments as middle state because we are only interested in the transition from recommended comments to netizens’ comments.

Fig. 5
figure 5

Three-state survival model of time effects on comment exposure transition probability from Recommended Comments exposure to Netizens’ Comments exposure

We show the predicted transition probabilities in Fig. 6. The white area in each figure represents the probabilities of respondents choosing recommended comments over time, whereas the gray area refers to the probabilities of them choosing netizens’ comments over time. From all four diagrams, we see that immediately after the exposure to the first policy, the white area (probabilities of respondents staying in recommended comments) shrinks precipitously, while the gray area (probabilities of respondents staying in netizens’ comments) increases significantly. This implies that for respondents who chose to view recommended comments at the beginning, the probability of them continuing to choose recommended comments declines over time, while the probability of them changing their comment exposure to netizens’ comments increases over time.

Fig. 6
figure 6

Transition probabilities across comment types

The figures help visualize the importance of Hypothesis 2 that individuals who exposed themselves to pro-regime comments are less likely to continue consuming them but more likely to switch their comment exposure to critical comments. To conclude, the results reveal that individuals likely consume alternative information that deviates from official discourses, thereby calling into question the effectiveness of soft propaganda orchestrated by the authorities (Dubois & Blank, 2018).

Approval of Policies

To examine whether online comments succeed in shaping people’s policy support, we need to compare our respondents with individuals who have no comment exposure. Recall that Study 2 consists of 652 respondents who were allowed to select comments to view.Footnote 8 We included additional 601 respondents who were shown the same policy news but were not given a chance to view any comment. The latter respondents serve as the control group for comparison, while the former respondents the treatment group.Footnote 9 By comparing the policy support of these two groups of respondents (N = 652 + 601 = 1253), we will be able to detect the potential effects of comment exposure on policy approval.

Table 3 shows the demographic characteristics of the respondents in the treatment group and the control group. Both groups have a similar number of respondents. Male respondents are slightly underrepresented in both groups, while members of the Chinese Communist Party are overrepresented in the sample. Despite these observational differences, the standardized mean differences of all variables are less than 0.1, suggesting that the control and treatment groups are more or less balanced across the covariates.

Table 3 Profiles of respondents in the control group and the treatment group

Since the dependent variable is a binary variable (agree/disagree with the policy), we apply both the random-effects logistic and the logistic regression modelFootnote 10 to analyze the effects of different comment exposures on respondents’ approval of policies between the treatment group and the control group. The random-effects logistic regression specification is as follows:

$$\Pr (Z_{it} ) = logit^{ - 1} (\sigma_{i} + \rho D_{i} + \mu N_{t} + \delta \left( {D_{i} \times N_{t} } \right) + P_{it}^{^{\prime}} \partial + W_{i}^{^{\prime}} \varphi + \lambda_{it})$$

where \(Z_{it}\) denotes individual i's approval for a policy shown at time t,Footnote 11\(\rho\) is the coefficient on the treatment status \(D_{i}\), which takes a value of 1 if respondent i is in the treatment group and 0 otherwise, \(\mu\) is the coefficient on policy number \(N_{t}\), \(\delta\) is the coefficient on the interaction term between treatment status \(D_{i}\) and policy number \(N_{t}\), \(\partial\) is a vector of coefficients on policy type dummies \(P_{it}\),Footnote 12\(\varphi\) is a vector of coefficients on individual-level control variables \({ }W_{i}\), \(\sigma_{i}\) is individual i's intercept, and \(\lambda_{it}\) is the error term clustered at the individual level which is assumed to be independent and identically distributed.

Table 4 shows the regression results. The coefficient on the treatment status \(D_{i}\) is significantly negative, which implies that compared with the control group, people in the treatment group who have choices of comment exposure show significantly lower levels of approval of the policies. This finding implies that with the influence of online opinions, netizens are more likely to question and even disapprove of social policies implemented by the state. The coefficients on the policy number variable \(N_{t}\) and the interaction term of policy number \(N_{t}\) and treatment status \(D_{i}\) are insignificant, indicating that individuals’ policy approvals in both groups are not affected by the display order of policies.

Table 4 Respondents’ policy approval over time

Political attitude variables also seem to play some roles in determining individuals’ policy approval. The coefficients on variables Internal political efficacy, Collectivism, National pride, and Society satisfaction are significantly positive. The results show that people who think politics is hard to understand, who consider collective interests are prior to individual interests, who have greater pride in China, and who are more satisfied with the current society are more likely to support the policies.

Our variable of interest affects policy approval via the standalone term \(D_{i}\) and the interaction term \(D_{i} \times N_{t}\). It is instructive to visualize the combined effects of these two terms. Figure 7 shows the marginal effects of the treatment on policy approval. As shown in the figure, individuals in the control group tend to disagree with the policies over time while respondents in the treatment group—namely, those who have a choice to view comments—consistently show markedly lower approval of the policies, although the significant difference between the treatment and control groups gradually disappears as \(N_{t}\) increases.

Fig. 7
figure 7

Marginal effects of policy approval over time between the treatment group and the control group

To conclude, once individuals are provided with a chance to view different discourses, they not only tend to expose themselves to critical information but also are influenced by comments and become more critical of the underlying policies. The result is consistent with Hypothesis 3.


Some scholars contend that leaders in authoritarian regimes in the twenty-first century no longer rule by fear. Rather, they rule by information manipulation (Chen & Xu, 2017; Guriev & Treisman, 2019; Rozenas & Stukal, 2019). In particular, autocrats use social media to run “soft propaganda” to change citizens’ political preferences (Gunitsky, 2015). One way to achieve this is to intervene in online discussion by producing and amplifying pro-regime comments. Those messages are believed to shape public opinions in favor of the regime, as they help strengthen individuals’ trust in the authorities, thereby promoting regime legitimacy.

While the regime has a strong incentive to spread pro-regime online comments, netizens’ views on such comments remain less understood. Some scholars argue that information manipulation in authoritarian regimes does not always work. At times it may even backfire, because people in those regimes may resist the information controls (Pan & Siegel, 2020; Roberts, 2020). This article provides one of the first systematic investigations into social media comment manipulation in China and aims to contribute to the literature of resilience to information manipulation in authoritarian regimes.

Our findings suggest that the comment section on social media platforms performs an important information role; it helps social media users gain unorthodox perspectives on social events. In other words, it provides Chinese netizens with a source of alternative information under a highly regulated media environment. This illustrates that even a tiny space for critical voices may undermine the public’s policy support. Indeed, manipulating social media comments is a kind of soft propaganda that is more subtle and less direct compared with censorship that aims to withhold information from the public entirely. Nonetheless, our findings indicate that people can be resilient to this subtle manipulation as well. People are not passive recipients of state propaganda (Rosenfeld, 2018). Rather, they have the ability to distinguish different kinds of information and even have an appetite for information that deviates from the official discourse. Extant works on authoritarian media controls focus on resilience against censorship (Chen & Yang, 2019; Hobbs & Roberts, 2018; Wong & Liang, 2021). Our findings suggest that resilience against soft propaganda is also worth investigating.