The Kardashian Index: a study of researchers' opinions on twitter 2014–2021

The Kardashian Index was designed for comedic purposes and meant to highlight researchers with minimal scholar contribution but oversized social media influence. We sought to examine attitudes and understanding of the Kardashian index by conducting a retrospective observational study of tweets retrieved from the Twitter API, Academic Track. From July 30th, 2014 to May 1st, 2021, 5826 unique tweets containing the phrase "Kardashian index” or related search terms were identified. Interest in the Kardashian Index peaked around the time of publication (July 30th, 2014), though the metric received continued discussion. One hundred random tweets were analyzed to see if the conversation points were positive, negative, or neutral. A majority (29%) of the tweets were neutral. Twenty-three percent of the tweets addressed the user’s own K-Index value, while 21% and 17% of the tweets were either critical or joking, respectively. Only a minority of tweets are critical of or appreciate the humor of the Kardashian index. The majority discuss the term matter-of-factly. Although the Kardashian Index was created in a lighthearted manner, a more serious tone emerges, prompting questions about the shifting role of scholarly and public influence.


Introduction
The Kardashian Index (K-Index), a metric that rises based on an individual's twitter follower count, and falls based on cumulative lifetime citations, was proposed by Dr. Neil Hall in 2014 as a humorous effort to draw attention to individuals with large public influence, but relatively minimal scholarly impact (Hall, 2014). Since its inception as a measurement of celebrity status (i.e., individuals whose social media popularity exceeds scholarly influence), some academics have referenced this metric as a tool to evaluate researchers in certain domains, such as cardiology (Khan et al., 2020).

3
There are several shortcomings to the metric. First, that junior researchers, even those with tremendous publication records, may score high, simply because followers accrue faster than lifetime citations. Another limitation is that the ability to popularize or communicate science clearly, drawing large audiences, is itself a virtue, and this skillset may complement or enhance traditional research dissemination. Yet, despite these limitations, which the index's developer initially conceded (Hall, 2014), the Kardashian index has gained recognition in the academy and Twittersphere. The index has even inspired spinoffs, such as the Modified-Kardashian Index (MK-Index). (Ebrahim & Nader, 2015) Even amidst these shortcomings, the K-Index, which was devised with a sense of humor, has attracted substantial attention in the literature, earning approximately 180 citations (Hall, 2014). This may be due to a variety of elements, including a misinterpretation of the article's tongue in cheek nature, resulting in a needless emotional charge, or it could reflect a broader interest, such as the role of social media in academic medicine. Twitter, as well as other social media platforms, have developed into engines for scientific dissemination, networking, and discourse. According to a randomized trial, articles selected to be shared on Twitter resulted in an increased citation count, suggesting that social media platforms may be an effective strategy to increase academic influence (Luc et al., 2021). While this is not the purpose of this study, it is our intention to contribute to the conversation around academics and social media.
We sought to study the usage of the term, and attitudes towards, Kardashian index on social media (Califf, 2020;Daly et al., 2020). Our research design is adopted from Thelwall & Kousha, who researched attitudes towards the h-index on Twitter (Thelwall & Kousha, 2021). Similar questions will be asked in our research: (1) Is there an increase or decrease in interest pertaining to the K-Index on Twitter? (2) What are researchers' attitudes toward the K-Index?

Methods
Our research design of this study is adopted from the Thelwall & Kousha's analysis of the h-index (Thelwall & Kousha, 2021). As this study involved only messages posted publicly, it did not constitute protected health care information, and this study was not submitted to institutional board review approval.

Data selection
We accessed the academic research Twitter API database (v1.1) to retrieve and analyze relevant tweets to the K-Index. The data were retrieved on May 16th, 2021. The search terms we used were "K-index," "K index," "Kardashian-index," and "Kardashian index." Each search query was supplemented with the exclusion prompts, "-is:retweet" to filter out any retweets and "-solar" to filter out tweets related to the K-index for space weather. Tweets were extracted from July 30th, 2014 to May 1st, 2021. Because the research that defined the Kardashian Index was published on July 30th, 2014, we chose that day to start our search (Hall, 2014). To adequately replicate Thelwall & Kousha, we specified the search terms in English to avoid a heterogeneous dataset and used the software, Mozdeh (version: Academic Research Twitter) to extract the tweets (Thelwall & Kousha, 2021). To avoid obtaining inaccurate results, the data set had to be modified to remove any identical tweets. After deleting duplicates, we created a unique data set from which we collected 19,482 tweets.
After adding exclusion modifiers to our original search queries (e.g., -solar), we observed that a portion of the remaining tweets were about space weather, as forecasters use a scale known as the K-Index to quantify disturbances in Earth's magnetic field (K-index et al., 2021). We used certain words associated with the planetary K-Index to clean the data set (e.g., solar, geomagnetic, planetary). After removing any tweets that dealt with the planetary k-index, our record set amounted to 64% of the original (n = 12,427). Eighty-three entries were eliminated because they lacked a date and a tweet, making it unfeasible to determine the original date. Any tweets not mentioning the K-Index, or the other terms used in the search query, were also removed. The final tweet count was 5826 tweets.

Data interpretation
User's gender was identified by the software, Mozdeh, which compares usernames against a database of commonly (i.e., > 90% prevalence) used first names by males and females (Thelwall xxxx).
A single reviewer assessed the discussion topics of tweets with at least 15 retweets. The original tweet may have been something that the community mainly agreed with or believed was worth sharing; therefore, retweets were considered. Thirty-nine tweets met these criteria and were analyzed.
Also, in alignment with Thelwall & Kousha, 100 random tweets were examined to determine their key points of discussion (Thelwall & Kousha, 2021). Random tweets were selected with a function included in the software, Mozdeh. Tweets in this data set that addressed non-study-related topics (e.g., solar weather) were replaced with other tweets generated by a random number generator. Two separate reviewers coded whether the discussion topics were in a positive, negative, neutral, or another context using the coding descriptions (Fig. 1). For simple tweets that linked to a website or article, the coding team referenced these articles and assessed them as if they were an extension of the tweet. Many people, for example, tweeted links to articles or blog posts about the K-Index. The author's views expressed in the article on the K-Index were assessed. However, many tweets included both a point of view and a link to an article. In these circumstances, the user's tweet took precedence over the article links.

Statistical analysis
We used Microsoft Excel to organize data and calculate frequencies. We used R statistical software, version 3.6.1 to generate a time plot, as well as to calculate a Cohen's Kappa for interrater reliability. We did not seek the consent from the institutional review board since we utilized publicly accessible data. This exemption applies to Twitter data since it is publicly available (Rivers et al., 2013).
The single tweet with the highest number of retweets (n = 63) about the K-Index was on January 7th, 2019 (Table 1). The user with the most tweets regarding the Kardashian Index has 90. This user had no relation or authorship role in the original Kardashian Index publication. Of tweets that could be matched to a specific gender, 657 (26%) were made by women, while 1866 (74%) were made by men. Since its inception on July 30th, 2014, the number of posts concerning the K-index has declined drastically, as seen in   (Table 1). The unweighted Cohen's Kappa for two raters assessing attitudes regarding Kardashian Index was 0.766, indicating a substantial level of interrater reliability (Cohen, 1960;Landis & Koch, 1977). Any differences between the code sheets were subjected to a collaborative review to identify a consensus coding. Figure 1 shows that a plurality of 100 tweets picked at random were neutral (29%). Twenty-three percent of the tweets addressed the user's own K-Index value, while 21% and 17% of the tweets were either critical or joking, respectively. The remaining tweets were concerning calculations (3%), positive (3%), about another person's K-Index value (2%), discussed another application (1%) or alternative (1%).

Discussion
Since publication of the Kardashian index on July 30th, 2014, the metric had an initial surge in discussion before rapidly declining in volume over a relatively short period of time (Fig. 1). While the metric currently experiences phases of ebb and flow, the general conversation has lagged markedly behind the debut period. The index is a numerical value that rises based on an individuals' twitter follower count and falls based on their cumulative lifetime citations. Thus, the index implies that those with high scores are Kardashians, widely recognized figures without a track-record of meritorious science. The use of the term was initially provided as a joke, and to our knowledge the term has no empirical, external verification. When analyzing the results, we found that the majority of tweets discussed the K-index in a matter-of-fact term (29%). Yet, despite the humorous intent, we discovered that a select percentage of users may be misinterpreting this measure in a different light than it was created. Any criticism voiced regarding the K-Index, which was 21% of the measured tweets in this study, may imply that the measure was interpreted seriously by the user. This may be because users misunderstood the original paper's aim, or because they had real concerns regardless of the intention. Additionally, it is unknown how many tweets (23%) referencing the user's own K-Index number were given with a self-deprecating sense of humor, neutrality, or criticism, which may result in different overall percentages.
One potential reason a sizable percentage of tweets referenced a user's personal K-Index value is that the creator of the K-Index advocated for scientists to compute their own K-Index annually and post it in their Twitter profile (Hall, 2014) Although this communication was addressed towards the user's biographical portion of their profile, many readers may have also tweeted about their K-Index as a result of this messaging.
Additionally, it is important to acknowledge that the K-Index was not established as a robust statistic, as the inventor even acknowledges himself by mentioning the significant limitations of the metric and requests readers to refrain from making the paper's methods a proxy for his typical research rigor (Hall, 2014). Even assuming that the K-Index equation is technically correct, the formula fails to account for several factors, some of which are discussed below. Although proponents state in theory that most popular scientists on Twitter would also contribute the most to the scientific literature (Khan et al., 2020), there are several limitations to this assertion: (1) Junior scientists-even with stellar publication records-will require time to accrue citations; (2) Individuals working in hot fields with a large number of researchers will accrue more citations and faster than those performing equally meritorious work in smaller fields (Larsen and Ins 2010); and (3) Individuals that succeed in science communication will inevitably have inflated K-indices, given the social media arena in which scientific knowledge is often disseminated.
Moreover, citations are an imprecise indicator of research quality. Noted pitfalls include the fact that some individuals with high citation counts may come from consensus documents or reviews (Larsen & von Ins, 2010) and may not reflect the work's originality, plausibility, or value (Aksnes et al., 2019). Even if the ideal Twitter follower meritocracy were based on citation count, this would still not place the most astute or thoughtful comments in positions of prominence. Furthermore, followers can come from those with expertise in other fields, such as scientists who are prolific authors, entrepreneurs, or personalities capable of successfully communicating ideas. Citations are not the sole indicator of competency, making the K-Index equation unidimensional and missing important complexity of social networks. While acknowledging that the K-Index is a creative metric developed in jest, these shortcomings are noted for reasons of interpreting the metric as a credible measure of researcher influence.
What does the future hold for the Kardashian Index? Apart from humor, the K-Index appears to have limited utility as an academic metric. A recent article showed no difference in the quantity of citations among cardiologists when stratified by their K-Index, suggesting that the negative connotation attached to Kardashian status may be inaccurate (Kalra et al., 2021). Despite these limitations, the K-Index may be useful as a catalyst to facilitate further research and discussion on the role of social media in scientific dialogue and dissemination.

Limitations
Our study has two significant limitations. First, we cannot be for sure whether these data adequately represent the views of researchers. Although the K-Index is an academic author metric, an analysis of twitter must consider that many users may be journalists, trainees, or media researchers (Thelwall & Kousha, 2021). Second, our analysis was based on the subjective assessment of tweets, which are known for their brevity. We aimed to limit judgment mistakes by using two independent reviewers and a pre-defined code description followed by a combined valuation to solve any reviewing discrepancies.

Conclusion
Since its debut, the Kardashian index has been frequently discussed on twitter. This index is elevated based on social media followers and reduced based on career lifetime citations and makes the implication that popular twitter accounts with limited citation trajectory are potentially undeserving, misleading, or frivolous. Proponents of the K-Index may not understand that the measure was created in jest and that its scientific value as a qualified index is limited. In the future, we should strive for indexes that require careful consideration when evaluating an author's competency rather than relying on a numerical value to determine a researcher's impact and influence in science.