1 Introduction

Previous literature has examined the effect of internet disclosure on received donations, showing a positive association between internet disclosures and NPO donations (Gandía, 2011; Saxton et al. 2014; Saxton and Wang 2014). Although these papers focus on the use of NPO websites, recent papers show the increasing relevance of social media in organizations’ communication strategy (Lovejoy and Saxton 2012; Guo and Saxton 2014; Zahrai et al. 2022). In this regard, several studies have explored the relationship between social media usage and NPO donations. Some research suggests that NPOs reliant on donations tend to have higher levels of social media activity (Gálvez-Rodríguez et al. 2016; Campbell and Lambright 2019). Additionally, other studies have found evidence supporting a connection between social media usage and increased donations (Saxton and Wang 2014; Xiao et al. 2022).

Studies on the use of social media by organizations are interesting because of the differences in user behavior, who tend to behave in a more emotional and impulsive way (Hollebeek et al. 2014; Dwivedi et al. 2019; Zahrai et al. 2022). Nonetheless, to date, no studies have considered how sentiment on social media may affect the donations received by NPOs. Sentiment is understood as the level of polarity transmitted by a text (Kearney and Liu 2014), and sentiment analysis of NPOs’ messages on social media provides an interesting setting to examine how the use of social media by organizations affects user behavior.

In that sense, agenda-setting theory (Waters 2013; Zhang 2016), hierarchy of engagement theory (Lovejoy and Saxton 2012), and framing theory (Xiao et al. 2022) provide a sound theoretical background that provides support for the hypothesis that the sentiment transmitted by NPOs through social media may have an effect on donations, an effect that may be different depending on the type of message. Therefore, the aim of this study is to examine whether the donations received by NPOs are affected by the sentiment of their social media posts. In particular, we examine whether the sentiment transmitted by NPOs on Twitter influences donations.

To do so, we use a sample composed of NPOs belonging to the Non-Profit Times 100 (NPT100) for the period 2015–2019 and modify the Weisbrod and Domínguez Model of Giving by including variables that proxy for social media engagement: NPO network size, NPO activity on Twitter, and sentiment transmitted by NPO posts. We estimate an aggregate annual measure that proxies for the sentiment transmitted by the NPO’s messages for each year of the sample period. Furthermore, following the hierarchy of engagement classification formulated by Lovejoy and Saxton (2012) and Svensson et al. (2015), we calculate partial measures of sentiment based on this classification that are also incorporated into the model.

The paper makes three main contributions: First, it extends the previous literature on how the use of social media can affect user behaviors; as far as we know, this is the first paper that examines whether the sentiment transmitted by NPOs’ social media may have an effect on their fundraising effectiveness. In this regard, the paper extends prior research regarding the effect of social media on donations (Saxton and Wang 2014). Second, it contributes to the application of the hierarchy of engagement classification theorized by Lovejoy and Saxton (2012) by estimating sentiment measures based on this classification that are applied to economic models. Thirdly, the study uses the framing theory to connect the hierarchy of engagement theory with the agenda-setting theory. In this regard, framing messages through social media plays an essential role in the NPO’s establishment of the perception that followers may have regarding the relevance of specific issues. Likewise, framing messages is also important to achieve a higher affective bond among followers, which can in turn increase their level of commitment to the NPO’s objectives.

The paper has the following structure: After the presentation of the study, we develop the theoretical framework on which we base our hypotheses in Sect. 2. In Sect. 3, we explain the models used to test our hypotheses, the measurement of sentiment, and the sample composition. Section 4 shows the results obtained from our analysis, which are discussed in Sect. 5. Finally, conclusions are presented in Sect. 6.

2 Theoretical framework

2.1 Sentiment on social media and NPO donations

Previous studies have analyzed how NPO donations are associated with online information disclosure (Gandía, 2011; Saxton et al. 2014; Saxton and Wang 2014). Gandía (2011) measures the level of internet disclosure in a sample of Spanish NPOs and finds evidence that the level of disclosure is positively related to the future funds received by NPOs. Furthermore, Saxton et al. (2014) examine the relationship between donations and the level of internet disclosure of 400 US NPOs, and they find a significant association between donations and internet disclosure; furthermore, they find evidence that the disclosures about mission-related performance are more relevant than those referred to financial reporting.

We must note that information disclosure is not limited to NPO websites, and social media has increasing relevance in NPO communication strategies (Lovejoy and Saxton 2012; Guo and Saxton 2014; Zahrai et al. 2022). Social media can be particularly interesting for NPOs as a fundraising channel. Gálvez-Rodríguez et al. (2016) analyze the determinants of the use of Twitter by NPOs, and they find evidence that NPOs with a greater dependence on donations make a greater effort to use Twitter as a communication mechanism. Similarly, Cambpell and Lambright (2019) find that NPOs reliant on program service fees and government funding have lower levels of social media adoption. On the other hand, Lee (2020) finds that moderate usage of Facebook is associated with increased volunteering, showing that social media may be useful for stimulating episodic volunteering.

On the other hand, we must note that social media users tend to adopt more emotional and impulsive behavior (Hollebeek et al. 2014; Dwivedi et al. 2019; Zahrai et al. 2022), taking decisions that are not based on rational expectations about the reported information (Saxton and Wang 2014). In this line, Hollebeek et al. (2014) examine consumer brand engagement in social media brands, and they find an association between emotional dimensions and social media brands. Dwivedi et al. (2019) develop a model that outlines how emotional brand attachment with social media explains social media consumer-based brand equity. On the other hand, Zahrai et al. (2022) find that excessive consumers of social media are driven more by their implicit attitudes than explicit beliefs in their consumption, and they provide evidence of an inconsistency between conscious attitudes and actual behavior toward social media.

The consideration of the emotional dimension may be relevant to understand how the dissemination of information on the internet, particularly through social media, can impact de ability of NPOs to receive donations. In this line, the agenda-setting theory and the framing theory provide theoretical foundations for understanding the effect of sentiment conveyed through social media on donations (Zhang 2016; Xiao et al. 2022).

This agenda-setting theory posits that media and social media can influence the public agenda by determining which issues are emphasized and which are ignored (Waters 2013). In this context, the second-level agenda-setting theory addresses how media and other information sources of information influence the public’s perception of the importance of specific issues. Social media can play a significant role in shaping the public agenda by allowing NPOs to effectively highlight and promote their causes, projects, and achievements. By using social media to disseminate information about their activities and the issues they address, NPOs can influence the audience’s perception of the importance of their specific topics.

On the other hand, the second-level agenda-setting theory suggests that people tend to give greater importance to issues highlighted in the media and on social media. By strategically utilizing social media, NPOs can spotlight a particular issue or cause in order to influence the audience’s perception of the importance of that topic (Waters 2013). This can lead potential donors to consider the cause as more relevant and, therefore, be more willing to make donations.

In that sense, the second-level agenda-setting theory is linked to the framing theory (Erlandsson et al. 2018; Baek et al. 2019; Xiao et al. 2022). This theory refers to how information is presented in framed to influence the perception of a specific topic or issue. In the context of NPOs using social media, framing becomes an essential tool for highlighting and promotion their causes. When NPOs frame their messages effectively on social media, they can influence how the audience perceives the importance of their projects and issues, thus affecting their donations.

In line with these theoretical foundations, previous literature has demonstrated that news media play a pivotal role in agenda-setting by determining how and at what level individuals donate to specific causes (Chapman et al. 2023). In this line, Waters (2013) finds evidence of a positive effect of news about natural disasters on donations when the news explicitly mention the involved organizations. Furthermore, Jones et al. (2018) find evidence that negative media stories about NPOs adversely affect their fundraising capabilities. In the context of social media, Xiao et al. (2022) provide evidence that framing of messages through social media have influence in donation intentions.

With regard to the association between emotional behavior in social media and donations, Saxton and Wang (2014) examine the donations made through social media, specifically Facebook Causes, and their results suggest that donations through social media are not determined by the same factors as in the traditional environment, suggesting a “social network” effect, where the main drivers of donations are more related to impulsive concerns than to rational ones. In this line, Galiano and Ravina (2021) examine the influence of emotions and social marketing on messages about volunteering on Facebook, and they find that the number of likes received by NPOs is associated with positive emotions in messages that talk about volunteering.

The results of these studies are linked to the hierarchy of engagement theory developed by Lovejoy and Saxton (2012), who classify the NPO tweets into three categories: (i) information messages; (ii) community messages; and iii) action messages. This classification determines the process by which the organization gets its supporters involved in the NPO’s projects: (i) In the first stage, the NPO tries to reach out to people through the dissemination of information of interest (information); (ii) in the second stage, the NPO tries to keep the flame alive by building an online community between the organization and the users (community); and (iii) as a last step, the NPO tries to convince its followers to step out to action, either through donations, volunteering activities, lobbying or participation in events (action).

Several studies have analyzed in more detail the hierarchy of engagement. Campbell and Lambright (2020) examine the factors that explain differences in engagement choices by human service organizations, and they find significant associations between organizational characteristics and social media content, showing that resource dependence urges NPOs to take more proactive behavior on social media. Furthermore, based on stewardship theory, they also find that social media may be helpful as a primary mode of engagement. On the other hand, Harris et al. (2021) also find a significant association between audience engagement (represented by countersignaling from users, such as likes, comments, and shares) and donations. In addition to audience engagement, Harris et al. (2021) also consider social media presence and organizational effort (proxied by the number of messages). Their results suggest that the three signaling social media dimensions affect donations, thus acting as substitutes for traditional fundraising expenditures.

The hierarchy of engagement theory can be combined with the framing theory in order to understand how NPOs can strategically use framing to achieve higher engagement and participation from their audience on social media. On the one hand, NPOs can strategically employ message framing in their social media posts to guide individuals through various levels of engagement in the hierarchy. On the other hand, NPOs can also tailor their message framing based on the level of audience engagement. By strategically using message framing and tailoring it to different levels of engagement, organizations can achieve higher engagement and mobilize their audience towards more meaningful actions, such as donations or volunteer activities in support of their causes.

Considering the relevance of social media engagement on donations, the more “emotional” behavior of users in social media, and how, according to the second-level agenda-setting theory and to the framing theory, the sentiment of messages can influence the emotional perceptions of social media users (Zhang 2016; Ceron et al. 2016; Xiao et al. 2022), we must bear in mind that the willingness of users may be affected not only by the information provided by the NPOs’ messages but also by the sentiment conveyed by this information. We define sentiment as the level of polarity that is transmitted from the reported text, i.e., whether the opinion expressed in the text is positive, neutral or negative, as well as other dimensions (Kearney and Liu 2014), and textual sentiment analysis refers to the use of techniques to identify and extract subjective and qualitative information from the texts under study (Gandía and Huguet 2021).

Considering the second-level agenda theory, framing theory, and hierarchy of engagement theory, we hypothesize that the sentiment expressed through social media can affect the level of donations received. Therefore, we formulate our first hypothesis:

H1:

The sentiment in NPOs’ social media posts has a significantly positive effect on the level of NPO donations.

Nevertheless, we must note that the association between sentiment in social media and donations may be affected by the type of messages: A message that calls followers to action is not going to have the same effect as one where the NPO is simply passing on Christmas greetings; similarly, the effect of the sentiment transmitted in these messages on donations may be different depending on the message’s category. Based on the hierarchy of engagement (Lovejoy and Saxon 2012; Svensson et al. 2015; Campbell and Lambright 2020), we set out separate hypotheses for the three categories of messages (i.e., information, community, and action), which we develop in the following subsections.

2.2 Sentiment in information messages and donations

The information category encompasses those messages whose main purpose is to provide information to the organization’s followers on social media. This generic category covers messages with very diverse content, ranging from information on the activities carried out by the organization to information on events or other issues of interest to the followers. Following Lovejoy and Saxton (2012), Svensson et al. (2015) separate information messages into those that refer to the NPO's programme and those about other matters.

In line with the subcategories proposed by Svensson et al. (2015), we consider that the association between sentiment and donations may be different depending on the subcategory. It is worth noting that, although both subcategories of messages aim to “inform” about matters relevant to followers, they actually differ in terms of the topics covered. In this context, the framing theory becomes especially relevant for studying the relationship between sentiment and donations. As stated by Erlandsson et al. (2018), negative charity appeals (i.e. advertisements emphasizing the bad consequences of not helping) can have a different effectiveness compared to positive charity appeals (i.e. advertisements emphasizing the good consequences of helping), and the strategic framing of messages may play a pivotal role in the effectiveness of these appeals.

With regard to messages about the NPO, positively-framed messages may emphasize the positive actions carried out by the NPO, which can increase the engagement with the NPO’s followers and make them more willing to participate in the NPO’s causes. Consequently, we would expect to observe a positive association between positive-framed messages’ polarity and the received donations. Therefore, we formulate our hypothesis related to this subcategory as follows:

H2a:

The sentiment in information messages about the NPO has a significantly positive effect on donations.

Regarding messages about other matters, they refer to the messages that the NPO posts but do not have a direct connection to its activities or program, meaning that the NPO is not the focal point of the message. In this subcategory, we can anticipate a different association, with a preference for negatively-framed messages (e.g. natural disasters, famines, or wars) that warn about the negative consequences of not helping. These negative messages are likely to induce NPO followers to engage in their causes and make donations. In this regard, previous literature has shown that negative sentiment has a higher influence than positive sentiment (Luo et al. 2023): Chang and Lee (2009) also find that negatively framed messages lead to greater behavioral intentional toward a campaign than positive messages, while Luo et al. (2023) find evidence that negative sentiment positively affects the volume, depth, and influence of information dissemination as compared with positive sentiment. Additionally, prior research has also provided evidence that news about natural disasters have a positive effect on donations when the news explicitly mentions the involved NPOs (Waters 2013). In this context, framing a story in negative terms may accentuate the dramatic nature of the message, appealing to the emotional behavior of potential donors and increasing their willingness to make donations.

Therefore, the negative sentiment in these messages prepares the audience for the call to action, thus having a positive effect on the donations received by the NPO. Thus, the relationship between sentiment and donations would be opposite to the one suggested for messages about the NPO: Given that negative-framed messages may have a positive effect on donations, the association between sentiment in messages about other matters and donations will be negative:

H2b:

The sentiment in information messages about other matters has a significantly negative effect on donations.

2.3 Sentiment in community messages and donations

Community messages are those whose main purpose is the creation and maintenance of an online community between the organization and its followers. These messages may include giving recognition and thanks, acknowledgment of current and local events, reply messages, and response solicitations. Community messages are crucial to obtain and maintain engagement with followers; as stated by Lovejoy and Saxton (2012), their purpose is “keeping the flame alive”. With regard to the association of the sentiment in community messages and donations, we consider their role in the hierarchy of engagement theory: Positive messages may increase the affective connection between NPOs and followers, who may decide to strengthen their ties with the NPO, thus having a positive effect on donations. Therefore, we formulate our hypothesis on community messages as follows:

H3:

The sentiment in community messages has a significantly positive effect on donations.

2.4 Sentiment on action messages and donations

Action messages have the purpose of encouraging the organization's followers to act in a certain way, such as promoting an event, donating, buying a product, or lobbying, among others (Lovejoy and Saxton 2012). With regard to the effect of sentiment in action messages, and based in the framing theory, we expect that negative-framed messages may be used by NPOs to enhance the impact of the messages, signaling the need or urgency for resources and the negative consequences of not helping, and providing a more dramatic connotation to increase the willingness to make donations. Given that the effect on donations may be driven by these negative-framed messages, the expected association between sentiment and donations is negative:

H4:

The sentiment in action messages has a significantly negative effect on the received donations.

Nevertheless, we have to note that, given that the action messages actually invite to action (i.e. they are action-framed), sentiment in action messages may not have a significant effect on donations, either because the sentiment may play a secondary role on donations, or because NPOs can use both negative- and positive-framed messages to call for action.

3 Research design

3.1 Model

We test our hypotheses with the following regression models:

$${LN\_DON}_{it}={\beta }_{0}+{\beta }_{1}{LN\_FUND}_{it}+{\beta }_{2}{LN\_PRICE}_{it}+{\beta }_{3}{AGE}_{it}+{\beta }_{4}{AGE*LN\_FUND}_{it}+{\beta }_{5}{FOLLOW}_{it}{\beta }_{6}+{FRIENDS}_{it}+{\beta }_{7}{TWEETS}_{it}+{\varepsilon }_{it}$$
(1a)
$${LN\_DON}_{it}={\beta }_{0}+{\beta }_{1}{LN\_FUND}_{it}+{\beta }_{2}{LN\_PRICE}_{it}+{\beta }_{3}{AGE}_{it}+{\beta }_{4}{AGE*LN\_FUND}_{it}+{\beta }_{5}{FOLLOW}_{it}{+\beta }_{6}{FRIENDS}_{it}+{\beta }_{7}{TWEETS}_{it}+{\beta }_{8}{POLARITY}_{it}+{\varepsilon }_{it}$$
(1b)
$${LN\_DON}_{it}={\beta }_{0}+{\beta }_{1}{LN\_FUND}_{it}+{\beta }_{2}{LN\_PRICE}_{it}+{\beta }_{3}{AGE}_{it}+{\beta }_{4}{AGE*LN\_FUND}_{it}+{\beta }_{5}{FOLLOW}_{it}{+\beta }_{6}{FRIENDS}_{it}+{\beta }_{7}{COM\_T}_{it}+{\beta }_{8}{ACT\_T}_{it}+{\beta }_{9}{INFO\_T}_{it}+{\beta }_{10}{POLARITY}_{it}+{\varepsilon }_{it}$$
(2a)
$${LN\_DON}_{it}={\beta }_{0}+{\beta }_{1}{LN\_FUND}_{it}+{\beta }_{2}{LN\_PRICE}_{it}+{\beta }_{3}{AGE}_{it}+{\beta }_{4}{AGE*LN\_FUND}_{it}+{\beta }_{5}{FOLLOW}_{it}{+\beta }_{6}{FRIENDS}_{it}+{\beta }_{7}{COM\_T}_{it}+{\beta }_{8}{ACT\_T}_{it}+{\beta }_{9}{INFO\_T}_{it}+{\beta }_{10}{ACT\_POL}_{it}+{\beta }_{11}{COM\_POL}_{it}+{\beta }_{12}{INFO\_POL}_{it}+{\varepsilon }_{it}$$
(2b)
$${LN\_DON}_{it}={\beta }_{0}+{\beta }_{1}{LN\_FUND}_{it}+{\beta }_{2}{LN\_PRICE}_{it}+{\beta }_{3}{AGE}_{it}+{\beta }_{4}{AGE*LN\_FUND}_{it}+{\beta }_{5}{FOLLOW}_{it}{+\beta }_{6}{FRIENDS}_{it}+{\beta }_{7}{COM\_T}_{it}+{\beta }_{8}{ACT\_T}_{it}+{\beta }_{9}{INFO\_T}_{it}+{\beta }_{10}{ACT\_POL}_{it}+{\beta }_{11}{COM\_POL}_{it}+{\beta }_{12}{INFO\_NGO\_POL}_{it}+{\beta }_{13}{INFO\_OTHERS\_POL}_{it}+{\varepsilon }_{it}$$
(2c)

These models are based on the model of giving used by Weisbrod and Domínguez (1986), which has been extensively used in previous literature (Marcuello and Salas 2000; Jacobs and Marudas 2009; Gandía, 2011; Tinkelman and Neely 2011; Saxton et al. 2014). The W&D model assumes that the demand for a particular collective good depends on its price, the quality of the good, and the information available to the buyer about both the price and the quality of the product (Gandía, 2011). Potential donors observe the good’s price (their contribution) but are uncertain about the quality of the good (the organization’s use of the donation). Therefore, organizations have incentives to provide information about the characteristics of their products, both through traditional channels, such as financial information (Christensen and Mohr 2003; Andrés-Alonso et al. 2006), and through the internet (Gandía, 2011; Saxton et al. 2014).

The dependent variable in the original W&D Model is the natural log of donations received by the organization (LN_DON), which is a proxy for the firm’s demand. With regard to the control variables used in the original Model, LN_FUND is the natural log of fundraising expenditure; it shows the positive effect that fundraising expenses have on donations, analogous to the effect that advertising expenditures have on sales in corporations. LN_PRICE is the natural log of the current year's price.Footnote 1 The variable proxies the cost to a donor of purchasing one euro’s worth of the organization’s output. The lower the price, the more efficient the NPO is at providing program services (Gandía, 2011). AGE is the NPO’s age, representing a proxy for the organization’s reputation, and AGE*LN_FUND is the interaction between AGE and LN_FUND, which reflects that additional expenses in fundraising will be less effective for well-established organizations.

In addition to the variables employed by the original W&D Model, and following Saxton and Wang (2014), Model [1a] also includes two variables related to the social network size: (i) the natural log of the number of followers (FOLLOW) and (ii) the natural log of the number of friends (FRIENDS) of the NPO’s Twitter account. These variables proxy for the social media network size; it is expected that NPOs with a greater social network receive higher donations. With regard to these variables, we have to note that historical data were not available, so we had to use the most recent data. Finally, we also include the natural log of the number of tweets published every year (TWEETS) as a proxy for the NPO’s activity on Twitter; we expect that more active NPOs will receive more donations. Model [1b] extends Model [1a] by including POLARITY, which represents the aggregate sentiment for an NPO’s tweets in a specific year. The measurement of this variable, as well as the other ones related to the sentiment of the specific categories (which are included in Models [2b] and [2c]), is explained in Sect. 3.2.

With regard to Model [2a], and in line with our hypotheses in Sect. 2, we consider that there may be differences in the impact of the NPO’s activity (proxied by TWEETS) depending on the category of the messages. Therefore, Model [2a] splits TWEETS into three variables (ACT_T, COM_T, and INFO_T) that represent the number of tweets in the action, community, and information categories. Similarly, Models [2b] and [2c] include the decomposition of POLARITY into several categories: Model [2b] includes three variables (ACT_POL, COM_POL, and INFO_POL) that show the sentiment in action, community, and information messages. Model [2c] goes one step further and decomposes INFO_POL into two variables: sentiment from messages about information on the NPO (INFO_NPO_POL) and sentiment from messages about information on other matters (INFO_OTHERS_POL).

3.2 Measurement of textual sentiment

The Models explained in Sect. 3.1 include test variables that proxy for the sentiment in the NPOs’ Twitter accounts, so we need to measure the sentiment/polarity on the NPOs’ messages. Sentiment analysis is performed through two main approaches (Gandía and Huguet 2021): (i) the use of dictionaries and (ii) machine learning or natural language processing (hereinafter NPL). Sentiment analysis through dictionaries is based on the classification of words, phrases or sentences from the document that are to be examined on the basis of predefined categories (Li 2010a). Documents are considered bags of words with an associated semantic orientation (Goel and Uzuner 2016). Machine learning is based on the application of one or more algorithms that “learn” from a training sample, which has been manually examined to identify the sentiment contained in the sample documents. Once the algorithm has examined the sentiment patterns in the training sample, it is applied to the entire corpus to derive an index textual sentiment (Kearney and Liu 2014).

Comparing the two approaches, machine learning is harder and time-consuming to implement because the training set must be manually classified, but it can be used when there is no specific dictionary to the language or type of document that is to be analyzed (Li 2010a). Furthermore, unlike the dictionary-based approach, machine learning techniques do take into consideration the context of a sentence. Studies that have used machine learning show that its accuracy is usually higher than when using the dictionary-based approach (Li 2010b; Huang et al. 2014). In any case, the use of machine learning models requires validation by human coders.

Once the approach to conduct sentiment analysis has been decided, the next step is to decide on which dimensions to measure the sentiment present in the messages; a classification based on the polarity of the text (negative, positive, or neutral) has been a common approach to it (Kearney and Liu 2014; Loughran and McDonald 2016). To determine text polarity, we use Textblob, a Python library for textual data processing, which provides a simple API for machine learning tasks, including text classification and sentiment analysis. Textblob allows for the use of a pre-trained model that classifies texts as positive, negative or neutral. Consequently, it does not merely add or subtract points based on positive or negative words, but has the capacity to identify patterns beyond individual words, such as the overall tone of a specific text (in this study, a tweet). Depending on the polarity of the text, Textblob assigns a score (ranging from − 1 to 1) to the analyzed text (each of the tweets published by the NPOs). Tweets with scores below -0.05 are considered negative, tweets with scores higher than 0.05 are classified as positive, and those with scores between − 0.05 and 0.05 are considered neutral.

We must note that we cannot test the isolated effect of individual tweets on NPO income since NPOs’ financial information is reported on a yearly basis. Therefore, we need to transform the individual tweets’ sentiment of a specific NPO into an aggregate annual measure to allow us to match the variables from the models explained in Sect. 3.2 with the sentiment of the tweets. To do this, we estimate a standardized measure (Tetlock et al. 2008): The sentiment of a specific NPO over a year (POLARITY) is calculated as the sum of the sentiment of each of the tweets (both positive, negative, and neutral) from the NPO on a specific year, divided by the total number of tweets from the NPO during the year:

$$ POLARITY = \frac{{\sum Sentiment\_Tweet_{ij} }}{Total tweets} $$

However, as we explained in Sect. 2.1, the aggregate sentiment may not be significantly associated with donations, because the effect may depend on the specific category of the tweets. In that sense, as explained in Sect. 2.2, Lovejoy and Saxton (2012) classified tweets into three categories (information, community, and action). This classification has been commonly used in previous literature (Svensson et al. 2014; Svensson et al. 2015; Cambpell and Lambright 2020), so we also use this categorization. To classify the tweets into these categories, we created a set of keywords that are linked to each category, as shown in Fig. 1. Categories are defined as follows:

  1. 1.

    Information category: Tweets containing information about the organization’s activities, event highlights, or any other news, facts or relevant information to stakeholders. Their main purpose is merely to inform. Following Svensson et al. (2015), we classify these information tweets into 2 subcategories: (1) tweets that inform about the NPOs’ activity and (2) tweets that inform about other matters.

  2. 2.

    Community category: Tweets aimed at creating an online community between the organization and its followers, which are also classified into two subcategories: (1) gratitude tweets to volunteers and/or participants and tweets acknowledging local and current events; and (2) interaction tweets with Twitter users (either by responding to users, by requesting a response from users, or by mentioning them).

  3. 3.

    Action category: Tweets whose purpose is to encourage their followers to act in a certain way: (1) tweets encouraging to make donations or buy products; (2) tweets encouraging to participate in campaigns and events; (3) tweets encouraging to take part in volunteering; and (4) tweets explaining how to help (tutorials).

Fig. 1
figure 1

List of category keywords

Full size image

Once the categories are defined, we use the list of keywords to classify the tweets in a specific category. We must note that, based on the keyword list, some tweets may appear in more than one category. To avoid this problem, we give preference to some categories over others. Therefore, we classify a tweet in the community category when we find at least one word from the community list of keywords. Moreover, we classify a tweet in the action category when we find at least one word from the action list of keywords (and there are no words belonging to the community list). The rest of tweets are classified as information tweets: When the tweet mentions the NPO, it is classified as an information message about the NPO’s activity. If there are no references to the NPO, the tweet is classified as an information post that informs about other matters.

Hence, based on the classification of tweets into the three categories (information, community, and action), we also decompose the aggregate sentiment into three variables, referred to as each category (INFO_POLARITY, COM_POLARITY, and ACT_POLARITY). Following the procedure used for calculating the aggregated sentiment, we calculate sentiment in each of these categories as the sum of sentiment from all tweets in a specific category during a year, divided by the total number of tweets belonging to that category during the year.

In a further analysis, we also decompose INFO_POLARITY into two variables: INFO_NPO_POL (sentiment from messages about information on the NPO) and INFO_OTHERS_POL (sentiment from messages about information on other matters). We have to note that keywords-based classification is subjective to some extent as it relies on the researchers’ criteria, and while the application of a machine learning model would have been desirable, given the absence of a well-established and readily available model, we assumed that keyword-based classification was the most practical way to approach the categorization of over 550,000 tweets.

3.3 Sample

To test our models, we gathered data from the 100 largest US NPOs in terms of total revenue, which are listed in the Times NPT100 list. Several studies on NPOs in the United States have used this list (Jacobs and Marudas 2006; Marudas and Jacobs 2006; Lovejoy and Saxton 2012), which is explained by the relevance of these organizations in the US nonprofit sphere. The NPOs’ financial data are directly collected from the website https://www.thenonprofittimes.com/. This website provides access to the list of the largest 100 NPOs in the USA in terms of total revenues for each year since 2012. The list contains financial for each NPO, including information on total revenues received, donations, and fundraising expenses, among others. These data are used to calculate the variables LN_DON, LN_FUND, and LN_PRICE, as explained in Sect. 3.1. We have gathered financial data for the period 2012–2019. However, because of the limitations in social media data collection that we explain below, we limit our analysis to the period 2015–2019.

With regard to the data about social media sentiment and activity, we have gathered them from Twitter, which is explained by the relevance of Twitter in capturing people’s global reactions (Svensson et al. 2015; Chen et al. 2019; Neu et al. 2019). For the collection of the tweets, we first obtained the Twitter accounts of the NPOs using the 2016 NPT100 list. Of the 100 listed NPOs, we found Twitter accounts for 99 of them; 8 of them were listed in 2016 but were no longer listed in 2017. For these outgoing NPOs, we collected their financial data from their Form-990.

Once we had the Twitter accounts, we extracted the tweets that were used for the estimation of the sentiment. To do this, we used a Python programming language code specifically written to interact with Twitter's application programming interface (API), allowing us to download the last 3,200 tweets from each of the organizations to a relational database. The limit of 3,200 tweets means that the temporal horizon of the Twitter activity we are able to capture depends on the level of activity of each NPO: The downloaded tweets covered a period of between 1 and 9 years; therefore, we have a shorter sample period (fewer observations) for the more active NPOs. To mitigate this limitation and obtain a more complete database, we gathered data at two separate times: October 2017 and January 2021. This procedure allows us to obtain the total sample of tweets for the period 2017–2019, but the sample suffers losses for the period 2015–2016. We tackle this limitation by performing an additional analysis including only the period 2017–2019, which is shown in Sect. 4.3.

Table 1 shows the distribution of tweets according to the three categories described in Sect. 3.2. We can see that community tweets are the most common messages (52.28% of the total sample), followed by information tweets (41.43%) and action tweets (6.29%). These percentages are quite similar to those obtained by Svensson et al. (2015), with percentages of 52.90% for information tweets, 42.80% for community tweets, and 4.30% for action tweets. With regard to the community tweets, 61.83% of them have a neutral polarity; in the action category, the most frequent polarity is positive (45.54%), while information tweets have a more even distribution (44.15% positive, 33.02% neutral). Table 2 shows the descriptive statistics of the variables included in the models.

Table 1 Tweet distribution by year and category
Full size table
Table 2 Descriptive statistics
Full size table

4 Results

4.1 Preliminary analysis: Aggregate sentiment measure

Table 3 shows the correlation matrix between the variables under study. We can see high correlations between the variables from the original W&D Model. In line with Lovejoy and Saxton (2012), we also observe a significant positive correlation between LN_REV and ACTION_T.

Table 3 Correlation matrix
Full size table

Table 4 shows the regression results of the original W&D Model and Models [1a] and [1b]. With regard to the W&D Model, we can see that all the variables are significant. After the inclusion of the social media variables, we can see that FRIENDS is significantly positive. With regard to social media activity, TWEETS shows a significantly negative coefficient, suggesting that NPOs that have more active profiles receive fewer funds. We consider that these results may be driven by the effect of omitted variables, so we will examine them in more depth in the following regressions. Regarding Hypothesis 1, POLARITY is not significant in Model 1b, suggesting that the tone of the messages does not have an effect on donations received. Nevertheless, as explained in Sect. 2.1, we have to note that this lack of significance may be because the effect of the tweets’ polarity may depend on the category of the message.

Table 4 Regression results of models W&D. [1a] and [1b]
Full size table

Therefore, considering the unexpected results for TWEETS and POLARITY, we decompose both the number of messages and the messages’ polarity based on the hierarchy of engagement classification and run Models [2a], [2b], and [2c] in order to test Hypotheses 2–4. The results for these regressions are shown in Sect. 4.2.

4.2 Main analysis: sentiment decomposition

In this section, we run Models [2a], [2b] and [2c]: Model [2a] divides the total number of tweets into the three categories (i.e., community, action, and information); furthermore, Model [2b] considers the separation of the aggregate sentiment in the three categories (ACTION_POL, COM_POL, and INFO_POL). Model [2c] goes one step further, and the sentiment in the information category is split into sentiment from messages about information on the NPO (INFO_NPO_POL) and sentiment from messages about information on other matters (INFO_OTHERS_POL). Table 5 shows the regression results of these models.

Table 5 Decomposition of TWEETS_N and POLARITY
Full size table

We can see that when considering the number of tweets belonging to each category, ACT_T is significantly positive in the three regressions, while COM_T is also significantly positive in two of the three regressions. These results suggest that a higher number of tweets belonging to the community and action categories have a positive effect on the donations received by the NPOs. With regard to the community category, whose main objective is “keeping the flame alive”, a higher activity helps to strengthen the engagement with the NPO’s community and thus has a positive side effect on the received donations. We must note that some of these tweets are directly addressed to followers, who may develop an affective affiliation with the NPO and thus decide to strengthen their ties with the NPOs via donations. Regarding the action category, these tweets have specifically the aim of encouraging certain actions from their followers, donations being among these actions; therefore, the results confirm that tweets in this category are effective.

On the other hand, the results also show a significantly negative effect of INFO_T, suggesting that higher activity in the information category reduces the donations received. Although surprising, these results may be related to the polarity of the tweets, given that we expect that information tweets about matters other than the NPO may have an effect when they have a negative polarity.

Concerning Hypotheses 2a and 2b, which address the effect of sentiment in information messages, we can see that INFO_POL is not significant. Nevertheless, when we split this category into messages belonging to the NPO (INFO_NPO_POL) and other matters (INFO_OTHERS_POL), the results are slightly different: While INFO_NPO_POL remains insignificant, INFO_OTHERS_POL is significantly negative. Regarding Hypothesis 2a, the results for INFO_NPO_POL suggest that sentiment in messages containing information about the NPO does not have an effect on the donations received, thus not supporting this hypothesis.

However, concerning Hypothesis 2b, the results for INFO_OTHERS_POL show that the sentiment in other matters is negatively associated with donations, supporting this hypothesis. We interpret the different effects between NPO-related messages and other matters-related messages issues in the context of framing theory: in the case of messages about other matters, NPOs frame them negatively to express concern about dramatic events and the negative consequences of inaction, encouraging followers to increase their donations.

With regard to Hypothesis 3, which deals with the effect of sentiment in community messages, the results indicate that COM_POL is significantly positive, suggesting a positive effect of sentiment in community tweets on the donations received and thus supporting Hypothesis 3. The results should be interpreted in accordance with the hierarchy of engagement theory: The positive sentiment in community messages can enhance emotional connection or positive affinity towards the NPO, strengthening ties with the NPO’s community and increasing the level of engagement among followers. Therefore, individuals are more likely to be willing to contribute financially to its cause.

Regarding Hypothesis 4, we can see that ACTION_POL is not statistically significant, suggesting that sentiment in these tweets does not have a significant effect on donations. Therefore, the results do not support our hypothesis. Although we expected a negative effect based on framing theory, the lack of significance of ACTION_POL may be related to the framing of action messages themselves. Since these messages inherently invite action (i.e. they are action-framed), sentiment in this category of messages may not have a significant effect, either because it plays a secondary role or because NPOs use both positive and negative sentiment to call for action.

4.3 Additional analysis: 2017–2019

As we have explained in Sect. 3.3, the data gathering process has the limitation that we were only able to download the last 3,200 tweets of each NPO on the gathering date. Although we tried to mitigate this limitation by gathering the data at two different times, sample losses remain for the period 2015–2016. To test the effect that this limitation has on our results, we perform an additional analysis by running the models from Sect. 4.2 for the 2017–2019 period. The results are shown in Table 6. We can observe that although some variables lose part of their significance, the results are qualitatively similar and support those obtained in Sect. 4.2: COM_T loses its significance in Models 2b and 2c, while ACTION_T loses its significance in Model 2a but remains significant for Models 2b and 2c, with the p value being approximately 10%; these variables, however, are not directly related with the stated hypotheses.

Table 6 Regression results for 2017–2019
Full size table

Nevertheless, sentiment variables (POLARITY, ACTION_POL, COM_POL, INFO_POL, INFO_NPO_POL, and INFO_OTHERS_POL) maintain the sign and significance of Sect. 4.1 and 4.2: POLARITY is not statistically significant, as in Sect. 4.1, thereby rejecting Hypothesis 1. Regarding the variables related to sentiment in information messages, INFO_POL and INFO_NPO_POL are not significant, while INFO_OTHERS_POL is significantly negative; these results are consistent with those in Sect. 4.2 and support Hypothesis 2b. The coefficient for COM_POL remains significantly positive, providing support for Hypothesis 3. Finally, the results for ACTION_POL continue to be non-significant, leading us to reject Hypothesis 4.

5 Discussion of results

Considering the results of Sects. 4.1, 4.2, and 4.3 as a whole, it can be observed that effect of sentiment in social media messages on donations is not homogeneous and depends on the type of message. In this regard, the agenda-setting theory, the framing theory and the hierarchy of engagement theory prove to be particularly relevant for understanding the obtained results. In Sect. 4.1, we observed that considering the sentiment of messages as a whole, without taking into account the framing provided by the message, does not yield significant results.

Results for information messages in Sect. 4.2 and 4.3, however, highlight the importance of framing: while sentiment in messages about the NPO do not have a significant effect, messages about other issues show a negative association driven by the positive effect of negative-framed messages on donations. These results are linked to those of Luo et al. (2023) and the agenda-setting theory: given that they find evidence that negative sentiment has more effect on the influence information dissemination, framing of messages in a negative tone may help NPOs to affect the public’s perception of the relevance of specific issues. Chang and Lee (2009) also find that negatively framed messages lead to greater behavioral intentional toward a campaign than positive messages. Therefore, the link between agenda-setting theory and framing theory may explain the negative association between sentiment in information messages about other issues and donations.

The results for action messages in Sect. 4.2 and 4.3 also demonstrate the relevance of framing: although sentiment in action messages does not have a significant effect, it is essential to emphasize that the number of action-oriented messages (i.e. frames as action messages) does have a positive effect on donations. These results can be related to the claims of Erlandsson et al. (2018) regarding the comparison between positive charity appeals and negative charity appeals and how the effectiveness of these messages depends not only on the framing of the messages themselves but also on how the measurement of effectiveness is operationalized. In that sense, it is worth noting that action messages actually invite to action, and thus the lack of significance of sentiment on donations may be due to sentiment playing a secondary role in this category of messages, or because NPOs use both positive and negative messages when calling for action.

On the other hand, results for community messages in Sects 4.2 and 4.3, showing a significantly positive effect of POL_COM on donations received, find support in the hierarchy of engagement theory and framing theory. As stated by Lovejoy and Saxton (2012), once attention has been garnered via information messages, the construction of an online community through community-framed messages is an essential step to increase the level of engagement of followers as a prerequisite to the call to action. In this regard, framing community-oriented messages in a positive tone can contribute to enhance the emotional bond between NPOs and followers, increasing their level of commitment with the goals of the NPOs. This, in turn, can increase followers’ willingness to contribute to the NPOs’ causes, resulting in an increase in donations received.

The study makes a triple contribution to the existing literature: firstly, it contributes to the examination of the impact of social media usage on donor behavior. In this regard, it incorporates the analysis of sentiment in social media messages, expanding on previous studies that have used measures of social media usage as determinants of donations models (Saxton and Wang 2014). Secondly, the study contributes to the application of the hierarchy of engagement theorized by Lovejoy and Saxton (2012) by estimating sentiment measures based on this classification, which are then applied to economic models. Finally, the study links framing theory with the hierarchy of engagement theory and agenda-setting theory. In this regard, the study demonstrates that framing social media messages can affect both followers’ perception of the relevance of certain issues (linked to agenda-setting theory) and the level of engagement among followers (linked to hierarchy of engagement theory).

6 Conclusions

We have examined whether the effect of sentiment transmitted on social media by NPOs affects NPO donations. Although a preliminary analysis shows that the variable that proxies for the aggregate sentiment of the NPO does not have a significant association with the donations received, we must note that the type of message may affect how sentiment is associated with donations. Based on the hierarchy of engagement classification, we separately examined the effect of sentiment of each category (information, community, and action). The results of this analysis show that sentiment linked to the community category has a positive effect on donations, while the sentiment of information messages about matters other than the NPO has a negative association, and the sentiment in action messages has no significant association.

We consider that the results are related to the second-level agenda-setting theory and the engagement theory, both connected in turn to framing theory. From the perspective of agenda-setting theory and framing theory, it is possible to explain the negative relationship between sentiment in messages about other issues and donations by using a negative framing in the messages. This framing seeks to influence public perception regarding the importance of the issues addressed and the potential adverse consequences of not taking action, which may stimulate an increase in donations. The results for action messages also demonstrate the relevance of framing: while the sentiment in these messages does not have a significant effect, the number of messages categorized as action does have a significantly positive effect. Considering that the framing of the message depends not only on the tone (positive vs negative) but also on its purpose (information, community, or action), framing messages with the action category proves to be an effective way to increase donations.

On the other hand, the positive effect of sentiment in community messages on donations finds theoretical support in engagement theory and framing theory: framing community messages with a positive tone increases the affective bond between the NPO and donors, which can increase their level of engagement and therefore affect their willingness to contribute to its causes, resulting in an increase in donations. The results on the variables that proxy for social media network size and social media activity support the hierarchy of engagement theory, suggesting that emotional features have significant relevance in social media.

The results have practical implications for both NPOs and donors. With regard to NPOs, it is worth noting how sentiment may affect the framing of the messages, thereby influencing both the perception that followers have of the NPO’s causes and the level of engagement of the followers. Therefore, the strategic framing of messages may play a pivotal role in the effectiveness of social media as a fundraising tool, complementing traditional fundraising campaigns. Regarding donors, the results show that their behavior is influenced by emotional factors, both through the framing of messages and through the emotional bond created by the NPO, which increases their level of engagement. Therefore, they should be aware of the ability of NPOs to manage their emotions via social media.

The study presents certain limitations regarding the measurement of textual sentiment. Firstly, the use of Textblob entails the utilization of a pre-trained machine learning model that does not consider the specific context for which it is being applied, i.e. NPOs’ messages in social media. In this regard, the development of an ad-hoc model that would take into account the particular characteristics of text in social media would have been more appropriate. Secondly, the measurement of polarity in this study represents only one type of measure of the media content sentiment. In this regard, other measures take into account not only the polarity (positive, negative, or neutral) of the text but also its relevance (Zhang 2016), as well as the utilization of a composite measure that integrates sentiment, visibility, and recency (Zhang 2018, 2019). Thirdly, the keywords-based classification has a subjective nature as it relied on the researchers’ criteria.

The study presents several opportunities for future research. First, the association between sentiment and received donations found in Twitter should be explored in other social media, such as Facebook. It is important to the extent that the donors’ profile may be more prone to certain media. Furthermore, future studies could address the limitations in the measurement of the sentiment variable by the development of an ad-hoc machine learning model for NPOs’ social media, as well as the calculation of measures that take into account not only the sentiment, but also other attributes, such as visibility. On the other hand, we have to note that we limit our analysis to text, but some social media (such as Instagram) are more focused on other content, such as images or videos, so the analysis of the impact that non-textual language may have on user behavior is relevant.