Impact of social network structure on content propagation: A study using YouTube data

Abstract

We study how the size and structure of the local network around a node affects the aggregate diffusion of products seeded by it. We examine this in the context of YouTube, the popular video-sharing site. We address the endogeneity problems common to this setting by using a rich dataset and a careful estimation methodology. We empirically demonstrate that the size and structure of an author’s local network is a significant driver of the popularity of videos seeded by her, even after controlling for observed and unobserved video characteristics, unobserved author characteristics, and endogenous network formation. Our findings are distinct from those in the peer effects literature, which examines neighborhood effects on individual behavior, since we document the causal relationship between a node’s local network position and the global diffusion of products seeded by it. Our results provide guidelines for identifying seeds that provide the best return on investment, thereby aiding managers conducting buzz marketing campaigns on social media forums. Further, our study sheds light on the other substantive factors that affect video consumption on YouTube.

Appendix

1.1 A.1. Technical details on data collection

We collected the YouTube data using a set of custom scripts written in Perl. We bootstrapped the data collection process using a Perl script to find the list of newly uploaded videos to YouTube. We then used a separate script to periodically access the statistics page corresponding to the videos, collected the relevant video characteristics, and stored them in a MySql database for later analysis. The Perl script parsed the HTML content of the statistics pages by looking for key markers in the HTML tags associated with the various video related data. We used Perl’s “HTML Parser” library to perform the data extraction.

Concurrently, we used a separate set of Perl scripts deployed on a cluster of workstations to collect data on the social network of the authors that have seeded the videos. The video page provided the link to the author’s page, which in turn contained data on the author and the author's social network. For instance, the author’s page contains the identities of his or her directly connected friends. We used a cluster of workstations in order to collect a snapshot of the social network structure within 4 days. The entire process was managed by a centralized controller that was responsible for handing out the network crawling tasks to the individual computers, monitoring their progress, and occasionally reissuing tasks if they are not completed within a specified time interval. The social network data was also stored in a MySql database and then analyzed using custom programs written in C. The analysis yielded the various social network metrics that we use in the paper, e.g., degree, number of second-degree friends, clustering, and Betweenness centrality.

We make all of the above scripts available for researchers interested in collecting YouTube data, at http://faculty.gsm.ucdavis.edu/~hema/youtube/. We do note that YouTube changes its webpage layout and data format regularly, so it is likely that our scripts would have to be modified to account for recent changes.

1.2 A.2. Initial conditions assumption

This assumption ensures that the impact of the unobserved fixed effect η _i on growth (y _i,t ) remains constant over time. Let ${A_K} = \sum\limits_{k = 1}^K {{\alpha_k}}$ and ${\eta_0} = \frac{{(1 + {\kappa_X}\gamma + {\kappa_Z}\beta )}}{{1 - {A_K}}}$. Recall that X _i,t and Z _i are linearly correlated with η _i . Hence, they can be expressed as ${X_{i,t}} = {\lambda_X}{X'_{i,t}} + {\kappa_X}{\eta_i} + {\delta_{i,t}}$ and ${Z_i} = {\lambda_Z}{Z'_i} + {\kappa_Z}{\eta_i} + {\xi_i}$, where ${X'_{i,t}}$ and ${Z'_i}$ is not correlated to η _i and δ _i,t and ξ _i are random shocks such that E(δ _i,t ) = 0, E(ξ _i ) = 0 and E(δ _i,t .η _i ) = 0, E(ξ _i .η _i ) = 0. Using these expansions recursively in tandem with Assumption 10.7, we now show that the impact of η _i has a constant y _i,t in all periods.

Lemma 1: The effect of the unobserved fixed effect η _i on y _i,t is constant for all periods and is equal to η ₀.

Proof: Period 1: Consider the growth equation for t = 1. By Assumption (10.7a), we have:

$$ {y_{i,1}} = c + {\eta_i}\left( {\frac{{1 + {\kappa_X}\gamma + {\kappa_Z}\beta {A_K}}}{{1 - {A_K}}}} \right) + \beta {Z_i} + {\varepsilon_{i,1}} $$

We know that Z _i can be expressed as ${Z_i} = {\lambda_Z}{Z'_i} + {\kappa_Z}{\eta_i} + {\xi_i}$, where Z’ _i and ξ _i are not correlated with η _i . After substituting for Z _i , (10.7a) can be expressed as follows:

$$ {y_{i,1}} = c + {\eta_0}{\eta_i} + {\lambda_Z}\beta {Z'_i} + \beta {\xi_i} + {\varepsilon_{i,1}}. $$

Since ${Z'_i}$, ξ _i , ɛ _i,1 and c are independent of η _i , the coefficient of η _i is given by η ₀.

Periods 2 to K: Next, consider the growth equations for the remaining (K-1) initial periods, i.e., 2 ⩽ t ⩽ K. From Assumption (10.7b), we have:

$$ {y_{i,t}} = c + {\eta_i}\left( {\frac{{1 + ({\kappa_Z}\beta + {\kappa_X}\gamma ){A_K}}}{{1 - {A_K}}}} \right) + \gamma {X_{i,t - 1}} + \beta {Z_i} + {\varepsilon_{i,t}}{, }\forall { }2 \leqslant t \leqslant K $$

As before, we can substitute for Z _i in (10.7b). In addition, we can also substitute for X _i,t−1 as follows: ${X_{i,t - 1}} = {\lambda_X}{X'_{i,t - 1}} + {\kappa_X}{\eta_i} + {\delta_{i,t - 1}}$). Thus, (10.7b) can be rewritten as:

$$ {y_{i,t}} = c + {\eta_0}{\eta_i} + {\lambda_x}\gamma {X'_{i,t - 1}} + \gamma {\delta_{i,t - 1}} + {\lambda_Z}\beta {Z'_i} + \beta {\xi_i} + {\varepsilon_{i,t}}{, }\forall { }1 < t \leqslant K $$

Since ${X'_{i,t - 1}}$, ${Z'_i}$, δ _i,t−1, ξ _i , ɛ _i,t and c are not correlated with η _i , the coefficient of η _i is given by η ₀.

Period K + 1: Next, consider the growth in (K + 1)^th period (from Equation 10)

$$ {y_{i,K + 1}} = c + \sum\limits_{k = 1}^K {{\alpha_k}{y_{i,t - k}}} + \gamma {X_{i,K}} + \beta {Z_i} + {\eta_i} + {\varepsilon_{i,K + 1}} $$

The η _i term in $\gamma {X_{i,K}} + \beta {Z_i} + {\eta_i}$ is given by $(1 + {\kappa_X}\gamma + {\kappa_Z}\beta ){\eta_i}$. We know that each y _i,t−k term in $\sum\limits_{k = 1}^K {{\alpha_k}{y_{i,t - k}}}$ contains η ₀ and therefore, the total contribution $\sum\limits_{k = 1}^K {{\alpha_k}{y_{i,t - k}}}$ to the η _i term is A _K η ₀ η _i . Thus, the complete coefficient of η _i in Equation (10) is given by η ₀.

Periods K + 2 to T: Now consider the growth in the (K + 2)^th period:

$$ {y_{i,K + 2}} = c + \sum\limits_{k = 2}^{K + 1} {{\alpha_k}{y_{i,t - k}}} + \gamma {X_{i,K + 1}} + \beta {Z_i} + {\eta_i} + {\varepsilon_{i,K + 2}} $$

As before, the η _i term from $\gamma {X_{i,K + 1}} + \beta {Z_i} + {\eta_i}$ is $(1 + {\kappa_X}\gamma + {\kappa_Z}\beta ){\eta_i}$ and since all y _i,t−k s in $\sum\limits_{k = 2}^{K + 1} {{\alpha_k}{y_{i,t - k}}}$ contain η ₀ η _i , their contribution to the η _i term is A _K η ₀ η _i . Thus, the total η _i term in y _i,K+2 is η ₀ η _i , which is the same as the η _i term in ${y_{i,1}},...,{y_{i,K + 1}}$. Next, to show that the coefficient of ${\eta_i}$ in y _i,K+3 is η ₀, we use two facts: 1) the functional form of y _i,K+3 is the same as that of y _i,K+2, and 2) the coefficient of η _i in all the lagged terms ${y_{i,K + 2}},...,{y_{i,3}}$, is η ₀. So using the same technique as above, we can show that the coefficient of η _i in y _i,K+3 is also η ₀. Similarly, by recursive induction, the coefficient of y _i,K+j is also η ₀ for all j > 3. Thus, all y _i,t s can be expressed as follows:

$$ {y_{i,t}} = {\eta_0}{\eta_i} + {f_t}\left( {{{X'}_{i,t - 1}},...,{{X'}_{i,1}},{\delta_{i,t - 1}},...,{\delta_{i,1}},{{Z'}_i},{\xi_i},{\varepsilon_{i,t}},...,{\varepsilon_{i,1}}} \right) $$

1.3 A.3. Moment conditions for first-differenced equation

The first-differenced equations are given by:

$$ \Delta {y_{i,t}} = \sum\limits_{k = 1}^K {{\alpha_k}\Delta {y_{i,t - k}}} + \gamma \Delta {X_{i,t - 1}} + \Delta {\varepsilon_{i,t}} $$

(11)

We specify two sets of moment conditions, 12(a) and 12(b), for Equation 11. In Proposition 1, we show that these moment conditions are true.

Proposition 1: For ${\it{p}} = {\it{K,K + {\text1},}}...{\text{,t - 2}}$, $E({y_{{\text{i}},{\text{p}}}}.\Delta {\varepsilon_{{\text{i}},{\text{t}}}}) = 0$ and $E({X_{{\text{i}},{\text{p}}}}.\Delta {\varepsilon_{i,t}}) = 0$.

Proof: We start with moment conditions $E({X_{{\text{i}},{\text{p}}}}.\Delta {\varepsilon_{i,t}}) = 0$ where ${\it{p}} = {\it{K,K + 1,}}...{\it{,t - {\text2}}}$. From Assumption (10.4), we have $E({X_{i,t}}.{\varepsilon_{i,s}}) = 0{\text{ if s}} > t$ and $E({X_{i,t}}.{\varepsilon_{i,s}}) \ne 0{\text{ if s}} \leqslant t$. This implies that X _i,p is uncorrelated to Δε _i,t for all p ≤ t − 2. Next, consider the moment conditions $E({y_{{\text{i}},{\text{p}}}}.\Delta {\varepsilon_{{\text{i}},{\text{t}}}}) = 0$, where ${\it{p}} = {\it{K,K + 1,}}...{\it{,t - {\text 2}}}$. From Lemma 1, we know that all y _i,t terms can be written as follows:

$$ {y_{i,p}} = {\eta_0}{\eta_i} + {f_p}\left( {{{X'}_{i,p - 1}},...,{{X'}_{i,1}},{\delta_{i,p - 1}},...,{\delta_{i,1}},{{Z'}_i},{\xi_i},{\varepsilon_{i,p}},...,{\varepsilon_{i,1}}} \right) $$

From Assumption (10.1), we know that $E({\eta_{\text{i}}}.\Delta {\varepsilon_{{\text{i}},{\text{t}}}}) = 0$. Also, from Assumptions 10.1, 10.4, 10.5, and 10.6, we know that ${f_p}(.)$ is uncorrelated with Δε _i,t for all p ≤ t − 2. So $E({y_{{\text{i}},{\text{p}}}}.\Delta {\varepsilon_{{\text{i}},{\text{t}}}}) = 0.$

1.4 A.4. Moment conditions for level equation

The level equations when t > K are given by:

$$ {y_{i,t}} = c + \sum\limits_{k = 1}^K {{\alpha_k}{y_{i,t - k}} + \gamma } {X_{i,t - 1}} + \beta {Z_i} + {\eta_i} + {\varepsilon_{i,t}} $$

(10)

We specify two sets of moment conditions for Equation (10) (see Equations 13a and 13b). In Proposition 2, we show that these moment conditions are true.

Proposition 2: For $p = K,...,t - 1$, $E\left( {\Delta {y_{{\text{i}},p}}.({\eta_{\text{i}}} + {\varepsilon_{{\text{i}},{\text{t}}}})} \right) = 0$ and $E\left( {\Delta {X_{{\text{i}},p}}.({\eta_{\text{i}}} + {\varepsilon_{{\text{i}},{\text{t}}}})} \right) = 0$.

Proof: We start with $E\left( {\Delta {X_{{\text{i}},p}}.({\eta_{\text{i}}} + {\varepsilon_{{\text{i}},{\text{t}}}})} \right) = 0$. From Assumption (10.4), we have $E({X_{i,t}}.{\varepsilon_{i,s}}) = 0{\text{ if s}} > t$ and $E({X_{i,t}}.{\varepsilon_{i,s}}) \ne 0{\text{ if s}} \leqslant t$. This implies that X _i,p is uncorrelated to ε _i,t for all p ≤ t − 1 and by extension ΔX _i,p is uncorrelated with ε _i,t for p ≤ t − 1. From Assumption (10.5), we know that X _i,p s are linearly correlated with η _i , which implies that ΔX _i,t−1 is uncorrelated with η _i . Thus, ΔX _i,p is uncorrelated to both η _i and ε _i,t . Next, consider the moments $E\left( {\Delta {y_{{\text{i}},p}}.({\eta_{\text{i}}} + {\varepsilon_{{\text{i}},{\text{t}}}})} \right) = 0$. Recall that, for p ≥ K + 1, y _i,p can be written as ${y_{i,p}} = {\eta_0}{\eta_i} + {f_p}(.)$, where ${f_p}(.)$ is independent of η _i . Hence, Δy _i,p can be written as follows:

$$ \Delta {y_{i,p}} = {f_p}\left( {{{X'}_{i,p - 1}},...,{{X'}_{i,1}},{{Z'}_i},{\varepsilon_{i,p}},...,{\varepsilon_{i,1}}} \right) - {f_{p - 1}}\left( {{{X'}_{i,p - 2}},...,{{X'}_{i,1}},{{Z'}_i},{\varepsilon_{i,t - 1}},...,{\varepsilon_{i,1}}} \right) $$

Thus, the moment condition, $E\left( {\Delta {y_{{\text{i}},p}}.({\eta_{\text{i}}} + {\varepsilon_{{\text{i}},{\text{t}}}})} \right) = 0$, can be expressed as follows:

$$ E\left( {\left( {{f_p}\left( {{{X'}_{i,p - 1}},..,{{X'}_{i,1}},{\delta_{i,p - 1}},..,{\delta_{i,1}},{{Z'}_i},{\xi_i},{\varepsilon_{i,p}},..,{\varepsilon_{i,1}}} \right) - {f_{p - 1}}\left( {{{X'}_{i,p - 2}},..,{{X'}_{i,1}},{\delta_{i,p - 2}},..,{\delta_{i,1}},{{Z'}_i},{\xi_i},{\varepsilon_{i,t - 1}},..,{\varepsilon_{i,1}}} \right)} \right).({\eta_{\text{i}}} + {\varepsilon_{{\text{i}},{\text{t}}}})} \right) = 0 $$

We already know that ${f_p}(.) - {f_{p - 1}}(.)$ is not correlated with η _i for all p. Following Assumptions (10.1) and (10.4), it is easy to see that ${f_p}(.) - {f_{p - 1}}(.)$ is also uncorrelated to ε _i,t for all p ≤ t − 1. Therefore, $E\left( {\Delta {y_{{\text{i}},p}}.({\eta_{\text{i}}} + {\varepsilon_{{\text{i}},{\text{t}}}})} \right) = 0$. Finally, note that ${f_p}(.) - {f_{p - 1}}(.)$ is correlated with X _i,t−1, Z _i and y _i,t−k s for p ≤ t − 1 ensuring that Δy _i,p s are good instruments for all these terms.