Abstract
Studies suggest that popular music genres are increasingly discussed by cultural intermediaries in ‘legitimate’ or ‘highbrow’ terms, rather than merely ‘lowbrow’ commercial entertainment. In addition, popular music discourse as produced by such intermediaries has historically been decidedly masculine—a trait which tends to increase on par with legitimation. However, seeing that women are gradually gaining symbolic and numerical representation in popular music production, this may have been changing over the last decade(s). In this article, we assess how popular music discourse within a key music media outlet (Pitchfork) changed between 1999 and 2021. We use word embedding models—a novel technique in computational social science—to assess legitimacy and gender in the discourses used in 23,992 reviews, and how this varies between genres. We find four notable patterns. First, reviews increasingly use a discourse that legitimates popular music, while, second, also increasingly using more feminine terms. This does not, third, occur simultaneously; however, discourse is either legitimate or feminine. Finally, these patterns also differ based on which popular music genres are discussed. The overall pattern is consistently found in pop, electronic and experimental, but not in historically masculine genres rap/hip-hop, metal and jazz which seem rather resistant to discursive change.
Similar content being viewed by others
Avoid common mistakes on your manuscript.
Introduction
Cultural intermediaries, such as reviewers, play a prominent role in how cultural products are perceived and evaluated in society [1]. Even in an era of social media and a decline in audiences for general broadsheet media, people turn to cultural intermediaries to make sense of and engage with cultural products such as music [2, 3]. Besides the influence of numerical evaluations in terms of grades or scores for specific cultural products, cultural intermediaries produce, maintain and/or deconstruct specific discourses which substantially shape how people perceive cultural products and how cultural products and genres are hierarchically valued and canonized [3, 4]. In other words, the way intermediaries in media outlets classify and discuss cultural products, such as music, film or literature, provides the contours for how people in general understand and assign value to these forms of culture [2, 5,6,7,8].
For example, in her classic study on media representation of music genres, Binder [9] demonstrated how, in the late 1980s and early 1990s, rap music and metal music were both discussed in terms of their potentially harmful influence on America’s youthful audiences, yet with a distinctly different, racialized focus between the two genres. In a comparable study on the media representation of popular music, Van Venrooij and Schmutz [10] show that cultural intermediaries in a variety of international media outlets increasingly used terms in their reviews that were once exclusively reserved for ‘highbrow’ culture. For example, critics writing for broadsheet newspapers on popular music were found to increasingly focus on aspects valued within highbrow arts, such as the music’s alleged seriousness, complexity, originality, or ambiguity [10]. In line with findings on the rise of the cultural ‘omnivore’, i.e. the cultural consumer who effortlessly moves between highbrow and popular cultural products [11, 12], this shows popular music’s increasing perceived legitimacy and continued role in social distinction processes within the (new) media landscape [cf. 13, 14]. Moreover, it highlights that media framing of cultural products is rarely stable and always in flux. Empirically grasping how cultural products are classified in key media outlets can, in other words, tell us how cultural hierarchies are changing over time.
While such studies have provided valuable and compelling insights, they often, understandably so, rely on qualitative content analyses within limited cross-sectional samples. Moreover, this tradition of research has mainly focussed on the way cultural intermediaries have propagated the distinction between supposedly ‘legitimate’ and ‘illegitimate’—popular and/or explicitly commercial—discourses on cultural products. This perspective can be extended beyond this distinction and applied to other dimensions of the public perception of cultural products, such as gendered variation in how these products are discussed (see for example: [15,16,17]). If we want to capture the prevalence of such discourses while simultaneously studying whether and in which directions these are developing over time, novel methods from computational science may be useful to explore. Moreover, the breadth and scope of such automated analyses also affords a more detailed focus on how discourse generated by cultural intermediaries are differentiated based on genre categorizations, which are crucial to how popular music is generally understood and classified [18].
In this article, we use a novel technique—word embedding models—to explore how cultural intermediaries' discourse surrounding popular music, in terms of legitimacy and gender, has changed over time. We do this by analysing a corpus of 23,992 music reviews spanning 22 years (1999–2021), published by the highly influential online music reviewing platform Pitchfork. Practically, we explore whether reviewers use a more legitimate or illegitimate (popular) discourse and, simultaneously, whether this discourse has become more masculine or feminine. Considering the central role of genre categorizations in how music is understood and perceived [18], we further explore how these evolving discourses are differentiated based on genre.
Our study makes three key contributions. First, understanding cultural intermediaries' changing discourse is key to assess whether public perception of cultural genres is changing. Such changes can directly affect the institutionalization of culture, such as in educational programmes and/or (cultural) policy [19,20,21]. While previous studies have convincingly demonstrated that popular music (and popular culture more generally) is increasingly discussed using legitimate criteria [2, 10, 22,23,24], it is important to include genre as a potentially differentiating variable within these developments. Second and related, the last decade we have witnessed a surge in attention for gender inequalities in Northwestern societies and media representation more broadly. This occurred alongside other music industry-specific initiatives towards more gender equality (for example, the European Keychange project to increase the number of women working in the music industry and the #metoo movement since 2017). Therefore, we do not only study cultural intermediaries discourse in terms legitimately vis-a-vis illegitimacy—the primary focus in extant research—but also in terms of masculinity versus femininity [15, 25]. Seeing that perceptions of legitimacy, particularly in terms of status, and gender are often related, with masculinity regularly being tied to legitimacy, and vice versa [26,27,28,29,30,31,32], it is important to explore such axes in comparison. Third, our study makes use of a novel methodology—word embedding models—to capture changing discourses within a substantial corpus of textual data. While qualitative discourse analyses remain highly relevant for studies on media discourse, our study offers a first attempt to approach this differently with a different and substantially increased scope.
In what follows, we will first discuss the role of cultural intermediaries and how they influence public opinion on cultural products such as music. Second, we zoom in on two central aspects that play a role in how popular music genres are perceived more broadly: (1) differences in whether genres are perceived as ‘legitimate’, i.e. ‘highbrow’ cultural genres or not, and (2) differences in whether genres are perceived as ‘masculine’ or ‘feminine’. Third, we explain the analytic strategy for this article, including the usage of word embedding models to explore the corpus of Pitchfork reviews. After this, we discuss our results. Here we demonstrate that, overall, discourse on Pitchfork increasingly uses a legitimating and feminine discourse. This provides evidence that popular music is increasingly discussed in terms historically preserved for highbrow art forms, while also moving from a masculine to a more feminine discourse. This, however, substantially differs between music genres: some genres follow this broader line (pop, experimental and electronic), whereas others (metal and rap/hip-hop) remain discursively represented in masculine and illegitimate terms, and jazz remaining masculine yet legitimate. We conclude the article with a discussion of the broader theoretical repercussions of our findings and the study’s limitations, and discuss potential opportunities for the usage of word embedding models in future social science studies.
Media discourse on popular music
The way media products are reviewed by critics and journalists has consequences for how these are perceived in society. Like any discourse, however, cultural intermediaries’ ways of discussing media products are subject to change over time. Focussing on music specifically, studies have convincingly demonstrated that assessing the attention dedicated to certain music genres within media sources provides a good means to learn about evolving musical hierarchies. For example, based on a quantitative content analysis of newspaper articles, Schmutz [17] demonstrates that between 1955 and 2005 a growing share of broadsheet newspaper space in several Northwestern countries was dedicated to popular music genres: a sign of its increasing cultural legitimacy. Yet, the extent of media attention for popular music and its various genres is not the only source of information on changing media perception; it is important to assess the discourses used as well [33].
While conceptually ‘discourse’ has seen various uses since Foucault foregrounded the term in his work [cf. 34–36], we define discourses as “broad systems of communication that link concepts together in a web of relationships through an underlying logic” [33]. Discourse produced by cultural intermediaries is important to consider because they contribute to and reproduce the taken-for-granted vocabulary people need to make sense of cultural products. When certain associations often occur in media discourse, for example, ‘bubblegum pop’ and ‘superficial’, these associations are considered more ‘natural’ and are more easily activated cognitively than other associations such as ‘bubblegum pop’ and ‘masterpiece’. The importance of understanding dominant associations within discourses and how they change lies exactly in the perceived ‘naturalness’ of the associations embedded in discourses: they reveal how some ways of understanding and perceiving have gained cultural dominance over others, making media discourse an important aspect of perceived cultural and social hierarchies. As such, scrutinizing aspects of discourse specifically how concepts used by cultural intermediaries are connected or disconnected from each other can benefit our understanding of the common underlying logics used to make sense of (new) cultural products, and how dominant these are within the media landscape. In studying the content of media discourse generated on popular music, we hence focus on two central aspects that are related to social and cultural hierarchies: legitimacy and gender.
Media discourse on legitimacy
Over the last decades, the question whether a cultural product is considered ‘artistic’ or not has moved beyond the strict bounds of the high arts into popular culture. Whereas fields such as popular music and film have a history of being conceptualized as ‘mass’ or ‘popular’ media, undeserving of artistic acclaim, this has broadly shifted since the 1990s [17, 37,38,39,40]. By producing the discourse, critics, journalists and (social) media commentators play a crucial role in legitimating some fields (and certain genres within them) over others [41, 42]. Ample studies demonstrate that many genres of media have been able to acquire artistic legitimacy over time: music genres such as jazz and rock [18, 43], comics and graphic novels [44, 45], or indie video games [46, 47]. A crucial ingredient to these processes of legitimation is whether the discourse surrounding such products is changing [30]. However, this is not simply a struggle about which opinion is deemed ‘best’. Cultural sociological studies, particularly the work of Bourdieu [48], demonstrate that whether a genre is perceived as (il)legitimate bears considerable consequences for the symbolic power asserted through affiliating oneself with this genre: ‘legitimate’ tastes versus ‘popular’ tastes [2, 11, 12, 48].
The question on how genres within popular music are generally perceived has received ample attention by cultural commentators and scholars in the past decades. The first studies on music taste by Arnold [49] and members of the Frankfurter Schule [50] departed from the binary and normative notion separating ‘highbrow’ genres, such as classical, from ‘lowbrow’ genres, such as pop or rock. In the 1960s, we were able to witness the rise of the so-called ‘rockism’ discourse that offered a first escape from this binary [22]. The term rockism was first used in the 1980s as a critique against this newly risen discourse that celebrated rock music for its supposed sincerity/authenticity, rawness and anti-commercialism. This discourse was typified ideologically combining legitimate aspects (anti-commercialism, seriousness) and aspects considered illegitimate (rawness, celebration of the visceral) from the perspective of a highbrow, ‘disinterested’ aesthetic disposition [17]. In response and adjacent to the rise of the cultural omnivore and the increased dominance of ‘middle-brow’ tastes, cultural intermediaries started advocating for the legitimacy of commercial, mainstream pop music and adjacent subgenres. This later became known as the ‘poptimist’ discourse [51], which “simultaneously embraces the pleasure of listening to pop music while asserting its legitimacy through making it the object of criticism” [2]. As such, poptimism unapologetically celebrates the commercial aspects of popular music genres, which were historically considered in an antagonistic or ironic way by cultural intermediaries (encapsulated perfectly with the term ‘guilty pleasure’) [23]. However, increasing or decreasing notions of (il)legitimacy in cultural intermediaries discourse rarely occurs in a vacuum and the process of legitimation is often coupled with masculinization as well. Historical examples can be found in the increasing masculinization of novel writing [31] and screen writing [32], which happened while these fields gained legitimacy in the broader cultural landscape. Indeed, as indicated by Schmutz, “the symbolic boundaries that sustain musical hierarchies continue to be linked to social boundaries, including gender” [17]. This underlines the necessity to include both legitimacy and gender when studying changes in media discourse on music.
Media discourse on gender
Focussing on gender, ample research has demonstrated that women and men are differently represented as artists across music genres. Whereas men are dominant across musical and technical positions in genres such as jazz, rock, metal and rap, women are generally better represented in genres such as chart pop and folk, although often still limited to specific positions such as vocals [52,53,54,55,56,57]. As a consequence of these divisions, discourses within these genres have often found to be masculine, which aids in keeping symbolic boundaries between genres intact and making bridging attempts relatively challenging [27]. Indeed, the ‘rockism’ discourse that gained traction in the late 1960s has also been a notably masculine one and has been widely criticized for it [2, 22, 23]. For example, studies find that masculine discourses in rock, punk and metal music historically portray these types of music as a form of masculine rebellion against ‘mainstream’ sensibilities, which are perceived as feminine [26, 58, 59]. When women participate in such genres or associated behavioural styles that are considered masculine, they are often judged more negatively than their male counterparts [60,61,62]. Instead, group category traits—identifying or being identified as women—become dominant in the evaluation process: successful (sexualized) performances of femininity [29, 63, 64], rather than individual musical skills. As such, gender has become an entrenched aspect of discourses surrounding specific music genres and their fields [10, 62].
The importance of gendered discourses within genres not only has consequences for representation of gender groups in various genres, either as audience or as artists, but also for how legitimate these genres are considered. For example, pop music was historically considered—at least until the rise of ‘poptimism’ sentiments—generally pre-fabricated, inferior and simple, and hence tied to notions of femininity [58, 65], whereas genres that are increasingly legitimated, such as jazz and rock, have come to be associated with masculinity [43, 66, 67]. The question remains, however, whether the coupling between legitimacy and masculinity that Schmutz [17] found and that the rockism and poptimism discourses allude to [2, 22, 23], can also be found in popular music discourse (re)produced in the last two decades. As such, the research question that drives this study is: how has the discourse on music in terms of (il)legitimacy and masculinity/femininity changed over time? While assessing this, we also explore to what extent this discourse differs between music genres, and whether changes in both dimensions of discourse relate to each other.
Analytical strategy
To analyse changing media discourse on music, we focus on Pitchfork—one of the largest and most influential music reviewing platforms available online. The self-proclaimed “most trusted voice in music” [31] was founded in 1995 by Ryan Schreiber and in 2015 acquired by major media company Condé Nast. A focus on Pitchfork has clear empirical advantages. While the platform initially focussed on indie/rock music, it diversified over time and, in this way, included most major genres. Importantly, there is no apparent relationship between genre affiliation and receiving a favourable review or not [8, 68], which we also corroborated in our research. In addition, it allows us to analyse a long period of time. Finally, according to our data, Pitchfork reviews have been written by a relatively diverse pool of over 650 authors in the last 20 years. This makes the site a relatively stable critical source in terms of professional cultural intermediaries firmly familiarized with journalistic practices, yet offering substantial within-source variation.
From a theoretical viewpoint, Pitchfork is interesting as it is a dominant voice in the field of music production, distribution and consumption and, in this way, has a major influence on how artists and genres are perceived. Since the early 2000s, reviews on Pitchfork are deemed highly influential in the making (or breaking) of artists, leading to the so-called ‘Pitchfork effect’ in musicians’ careers, particularly for early career artists [8, 69]. Even in an era of increasing reviewing practices on social media such as TikTok and YouTube, content published on Pitchfork has considerable global reach, specifically through their own social media channels which have millions of followers. According to the website itself, they welcome “more than 7 million monthly unique visitors” [31]. This would place them above other well-known music reviewing websites such as Stereogum and Consequence of Sound, and slightly below giants such as NME, Billboard and Rolling Stone (which, unlike Pitchfork, all have a considerable history as paper magazines). Data from similarweb.com [70] shows that the gender distribution of visitors of Pitchfork is about 60% men versus 40% women, with most visitors being between 25 and 34 years old.
To analyse the media discourse on music in our corpus, i.e. the total collection of reviews, we use word embedding models. Word embeddings represent words as vectors in a shared vector space [71]. Put simply, words that often share similar contexts will produce similar word vectors and will, therefore, be closer to each other in this so-called ‘vector space’. Consider this vector space as a giant, n-dimensional, semantic space which includes all the words present in the collection of texts one is analysing. The more similar words are, the closer they will be to each other in this giant semantic space. For example, the distance between the words ‘guitar’ and ‘music’ will be smaller than the distance between the words ‘guitar’ and ‘climate’. How is this semantic space created? From linguistic theory, we know that the meaning of words can be inferred from the context in which they are used [72]. So, words that are used in similar contexts throughout the corpus are similar to each other in terms of meaning. Thinking specifically about the context of music reviews, consider, for example, the words ‘vocalist’ and ‘singer’. Both words may never co-occur in a review, as some reviewers may systematically prefer one word over the other, but they will share similar contexts as they are likely both surrounded by words, such as ‘voice’, ‘charismatic’, ‘falsetto’, or ‘nasal’. Because of the similarity in context, a word embedding model is able to detect that the semantic similarity between ‘vocalist’ and ‘singer’ is very high. In this way, word embeddings allow to capture complex semantic relationships integral to discourse.
Word embedding models have been used in computer science for text classification. They are, however, also extremely useful for social science as they are able to detect recurring associations in texts [72,73,74,75]. For example, Kozlowski et al. [73] use word embedding models to trace historical changes in the way social class is understood. Similarly, Nelson [74] uses word embeddings to map how black and white persons are discursively represented in the nineteenth century US South. Rodman [72] studies’ 160 years of newspaper coverage to track the changing meaning of political concepts. In line with these studies, we use word embeddings to track changes in the discursive representation of genres.
In this article, we are specifically interested in the (potentially) changing discourse on popular music on the poles of two oppositions: illegitimate vs. legitimate (as an indicator of legitimacy) and masculine vs. feminine (as an indicator of gender). To measure the engagement of documents with these poles, we use Concept Mover’s Distance (CMD) which was developed by Stoltz and Taylor [76, 77]. By extending the idea of similarity between two words, CMD measures the similarity between a document and a theoretically motivated concept. CMD holds “the embedding space constant while measuring the position of documents relative to it, specifically measuring each document’s distance from a specific ‘region’ of the embedding space” [75]. In this way, our analysis will track the relative position of documents towards each pole of both dimensions. We use a pre-trained word embedding space. More specifically, we use word embeddings from fastText which were trained on Wikipedia 2017, UMBC webbase corpus and statmt.org news dataset [78, 79] and which are known for their high quality [76].
Using CMD, we can measure the position of each review on the legitimacy dimension, which ranges from illegitimate to legitimate, and the gender dimension, which ranges from feminine to masculine. Consider, for example, a review that is close to the feminine pole of the gender dimension. This review may be full of negative gender stereotypes with sentences such as: ‘she offers a simple, feminine style to the guitar playing’. That review may also be located on the feminine pole of the gender dimension because of other things than gender stereotypes. For example, suppose a reviewer discusses an all-woman group and reflects on the role of each band member. The reviewer may use sentences like ‘she is very skilled in her guitar playing’ or ‘she is displaying her well-known style of singing’. The words ‘she’ and ‘her’ will strongly pull the review towards the feminine pole. This type of aspects picked up by the model are more ‘objective’, compared to gender stereotypes, in the sense that they relate to ‘objective gender divisions’ in music. That does not mean, however, that this is irrelevant for social sciences. Quite the opposite: the ‘objective’ gender division is also a concrete reflection of how genres are perceived in society. If genre x is, according to the dominant beliefs in a society, clearly a ‘feminine’ genre, it may prevent boys/men to listen to it, to develop a preference for it and, in the end, contribute to the genre by producing music themselves (as indicated by, for example: [56, 62, 66, 80, 81]).
So, in general, we aim to capture the discourses surrounding popular music in reviews on two dimensions: legitimacy and gender. While these discourses are obviously shaped by dominant beliefs in society (because the reviewers themselves are part of society), these representations also become strong drivers themselves. Once these beliefs are included in the discourse of a dominant player in the music field, which Pitchfork is, they become a strong force in further shaping the way genres are perceived.
Data and methods
Our data include 22 years of Pitchfork reviews, from 1999 to 2021 (available through: [82]). Our corpus consists of 23,992 reviews containing over 16,307,771 words. Panel A in Fig. 1Footnote 1 presents the number of reviews over time. Next to year of publication of the review, the music genre, to which the review is allocated to by Pitchfork, is central to our analysis. Pitchfork categorizes reviews into nine genres: electronic, folk/country, jazz, pop/R&B, rock, experimental, global, metal, rap/hip-hop. While some of these genre labels are rather ambiguous (‘global’, for example), we choose to use the genre categorization used by Pitchfork rather than developing a new genre categorization. This is firstly because of the inductive nature of our analysis and secondly because we are ultimately interested in the way music is represented by Pitchfork, including its chosen genre categories. The majority of reviews have only one genre label (n = 17,764). There are, however, also reviews with multiple genre labels. For example, reviews of albums from the artist Bonobo have the labels ‘electronic’ and ‘jazz’. In total, 3446 reviews have two genre labels, 349 reviews have three genre labels, and 31 reviews have four genre labels. There are also 2402 reviews without any genre label, which is often the case for compilation albums from various artists. The visualizations in “Results” pertaining to the entire corpus include all reviews, so also the reviews without a genre label. These reviews, however, are not included in any genre-specific visualization shown later in the analysis.
To prepare the reviews for the CMD analysis, we pre-process the data by lowercasing them and removing all punctuation and numbers. To run the analysis, we use the CMDist package in R [83].Footnote 2 For each document, the analysis calculates “the minimum cost that a document’s embedded words need to travel to arrive at the position of all the words in an ideal ‘pseudo document’ consisting of only words denoting a specified concept” [76]. This can also be used to measure the extent documents engage with opposing poles of a concept [77] (see also [73]). To capture both dimensions, we create a list of so-called ‘anchor words’ for each pole: masculine (e.g. ‘man’, ‘male’, ‘masculine), feminine (e.g. ‘woman’, ‘female’, ‘feminine), illegitimate (e.g. ‘lowbrow’, ‘unsophisticated’, ‘commercial’) and legitimate (e.g. ‘highbrow’, ‘sophisticated’, ‘art’). The selection of these anchor words is inspired by extant research on gender and legitimacy in the representation of music [10, 26, 27, 85, 86] and on the anchor words used by previous studies using word embeddings [74]. We also check the appropriateness of each anchor word by looking at the nearest ‘neighbours’ in the word embedding model [83]. In this way, we selected the anchor words which are presented in Table 1. This creates a one-dimensional continuum between masculine and feminine on which each document is situated, i.e. the CDM score for gender. Similarly, each document is positioned on a one-dimensional continuum between illegitimate and legitimate, i.e. the CMD score for legitimacy.
Discursive representation of music over time
First, we look at the evolution of discourse within all reviews, which is presented in Fig. 2. There are two clear upward trends over time. That is, music is increasingly represented through a more legitimate and a more feminine discourse. While we see a clear increment of both types of discourses, this does not automatically imply that both discourses correlate. That is, do reviews with a more legitimate discourse also include a more feminine discourse? To study this, we look at the relationship between the CMD score for legitimacy and the CMD score for gender.
To see how the legitimacy discourse and the gender discourse associate to each other, we fit linear models. While legitimacy is the dependent variable in our models, the models should not be interpreted causally. We are interested in the association between both types of discourse, while controlling for year and genres. Table 2 presents the two models that we fitted. In Model 1, we model the relationship between the legitimacy discourse and the gender discourse as a linear relationship. In Model 2, we include a polynomial to see whether the relationship is non-linear. Including this polynomial significantly improved model fit (F = 213,56; p < 0.0001).Footnote 3 To aid interpretation, we visualize the non-linear relationship between the legitimacy discourse and the gender discourse in Fig. 3.
Figure 3 clearly shows a non-linear relationship between the discourses regarding legitimacy and gender. Reviews that have a CMD score on the gender discourse around 0 have the highest CMD scores on the legitimacy discourse. In other words, reviews that rely on a legitimate discourse to a large extent are reviews that or not particularly gendered. On the other hand, reviews that are gendered particularly masculine or feminine are reviews that do not use a legitimacy discourse. It appears that reviewers make a sort of trade-off when they write their review: they either use a discourse focusing on the legitimacy of music (and shun from using a gendered language) or use a gendered discourse (and shun from discussing the legitimacy of the music). In additional analysis, which are not presented here, we find no relationship between these patterns and the numerical evaluations that reviewers offer on the discussed music.Footnote 4 In other words, none of these discourses directly relate to whether a review overall is positive or negative about the work. Tentatively, this seems to be in line with the rise of the poptimist discourse, which prescribes the inclusion of mainstream ‘popular’ sentiments, but does not necessarily imply that these are critically celebrated. We will elaborate on this in the discussion, but we first turn our focus to the role played by genre.
Discursive representation of different music genres
In addition to the general increment of both types of discursive representation, we are interested in the role of genres. How does each relate to the legitimacy discourse and the gender discourse? In Fig. 4, we plot the mean CMD scores for each year within each genre.Footnote 5 This is particularly interesting as it allows us to understand the relative position of the genres on both forms of discourse. Zooming in on this, we can identify three patterns.
First, for most genres, the general pattern applies. That is, for most genres there seems to be a trend towards legitimization and feminization. This is particularly the case for the genres pop, electronic and experimental (and rock, to a lesser extent) that were more at the masculine/illegitimate pole in early Pitchfork reviews and have increasingly moved towards the feminine/legitimate pole.
Second, in the folk/country genre, a more feminine discourse has dominated for the entire period of Pitchfork’s existence—the only genre in which this is the case. As with the above-mentioned genres, over the years, the albums from the folk/country genre have increasingly been discussed using a more legitimate discourse, while remaining at the feminine end of the gender spectrum.
Third, there are three genres which are located at the masculine end of the gender spectrum for each year: rap/hip-hop, metal and jazz. This is in line with previous scholarship on the homology between gender categories and these specific genres [52, 87,88,89]. Interestingly however, these three genres also score relatively consistently in terms of legitimacy. We find that rap/hip-hop and metal are predominantly discussed drawing from an illegitimate discourse throughout time (as also found in [9, 90]), while jazz is discussed using a legitimate discourse. Evidently, this can be related to jazz’s incorporation into the realm of ‘high’ culture, as a genre located aside classical and distinct from ‘popular’ music genres. Taking a step back, we can conclude that genres in which a distinctly legitimate (jazz) or illegitimate (rap/hip-hop, metal) discourse is used, are also discursively more masculine and rather resistant to change. For all the other genres, where the general pattern of femininization plus legitimization applies, we do not find such clear-cut distinctions. The most rigid genre discourses in terms of (il)legitimacy, then, are also the most masculine.
Discussion and conclusion
We used word embedding models, a novel technique from computational social science methods, to understand how the discourse on popular music in terms of (il)legitimacy and masculinity/femininity has changed over time. Based on a sample of over 20,000 reviews published on the music reviewing website Pitchfork between 1999 and 2021, we found four notable patterns. First, looking at the overall pattern, discourse in reviews of music on Pitchfork has gradually become more legitimate. This means that in reviews, reviewers more often draw from criteria historically reserved for ‘highbrow’ categories such as seriousness, originality or complexity. Second, this pattern of an increasingly legitimating discourse is accompanied with a decreasing masculine discourse, moving gradually towards including more feminine terms. Third, however, these changing discourses often do not happen simultaneously in the same reviews: reviewers seem to make a trade-off between using a legitimating discourse or a gendered discourse, but not often at the same time. When zooming in on specific genre categories as used by Pitchfork, we find, however, that, fourth, these patterns differ based on genre groups. Whereas the overall pattern towards a legitimating and feminine discourse is especially pronounced for the genres pop and electronic, this is not the case for historically male-dominated genres rap/hip-hop, metal and jazz. Whereas reviews of jazz music make use of a more masculine and legitimate discourse, reviews of music within the rap/hip-hop and metal genres make use of a masculine and illegitimate discourse. In comparison to the other genre categories used on Pitchfork, rap/hip-hop, metal and jazz seem particularly resistant to discursive change.
We offer a number of tentative theoretical interpretations of these findings. First, one overall explanation for the rise of a legitimate discourse overall is that cultural intermediaries in general tend to disentangle the ‘what’ and the ‘how’ of cultural consumption and increasingly apply an aesthetic perspective when discussing cultural products [2, 17, 88, 91]. Moreover, at least for consumers, digitalization of popular music and its resulting ubiquity has further moved familiar processes of distinction to realms of technical competence and (expensive) material sound carriers, rather than the music itself [13, 14]. This is in line with the general rise of cultural omnivores, which does not only mean that (critical) consumers commonly appreciate more different genres, but also view them through a ‘highbrow’ lens [8], while seeking distinction through other means.
Second, however, it is surprising that the usage of a legitimate discourse does not necessarily relate to receiving a high score for an album. An explanation for this may be that a discourse is used as a way of classifying, i.e. drawing genre demarcations and associating genres to other categories rather than a way of evaluating. Indeed, the ‘poptimism’ ideology that gained traction as a response to the ruling ‘rockism’ perspective in the 2000s may have shaped this situation, where albums that are discussed using generally illegitimate terms—blatantly commercial, simple and easy to digest—receive high evaluations for exactly that reason. Our findings indeed indicate the continued existence of both the rockism and the poptimism discourses, although it is notable that hip-hop seems to have joined the ranks of rock in the former. This is not surprising, however, as the appreciation for rawness and attitudes of “keeping it real” [22] is particularly pronounced in hip-hop as well [89]. It is then mainly the genres that are considered most commercial and mainstream—pop and electronic—that the intellectualizing poptimist discourse is pronounced and increasing in presence on Pitchfork. In summary, the fact that the usage of these discourses is not necessarily related to the scores attributed to albums is a case in point: it suggests that the discourse itself—the way reviewers talk about an album and/or an artist and the underlying logic they draw from—is related to the genre allocation of an album or artist, rather than to whether something is appreciated or not. However, it is important to note that a deeper, qualitative discourse analysis would be necessary to corroborate this theoretical interpretation.
Third, discourses regarding cultural legitimacy and gender may be context and/or field dependent, as is demonstrably so in the case with music genres. The associations grounded in these discourses also ‘live’ beyond music categories. Research on cognition and cultural categorizations demonstrate how entrenched such associations can be(come) and, as such, have consequences for exclusion as they are both intentionally and unintentionally shared in everyday classification processes [92,93,94]. For gender, in particular, this can mean that an artist’s gender can negatively or positively influence how they are perceived within the field that they desire to operate in, as previous studies have demonstrated [26, 62]. But for legitimizing discourses, this can have consequences as well: cultural products seen as culturally legitimate have higher chances of being offered governmental subsidies, being integrated into educational programmes, or other kinds of institutional embedding [19,20,21], further strengthening their perceived legitimacy. When coupled with social categories such as gender, as our research demonstrates, this can have multiplying consequences for sustainable inclusion. Our findings suggests that a legitimizing process has been unfolding for mainstream, (often) explicitly commercial music genres such as pop and electronic, that are connected to taste groups historically perceived as middle-brow [2, 88]. Coupled with the increasing discursive representation of femininity within these genres, it seems that—opposed to the rockism discourse that intellectualized and masculinized rock music criticism—the legitimation of pop and electronic is decidedly decoupled from entrenched gender patterns found for other cultural genres such as literature or screen writing. This means that, at least in terms of symbolic representation, women are generally more notably present in popular music discourse compared to the past, and that this is—at least on Pitchfork—unrelated to the numerical evaluations granted by reviewers.
Our article also contributes to social science by demonstrating the value of word embedding models in the study of discourse at least in its ‘light’, associative definition of underlying patterns in the words people use to describe things [33]. It is particularly useful to study differences in discourse [72, 74, 75] by, for example, empirically demonstrating how multiple discourses available in a society differ from each other or how discourses evolve over time. Such large-scale semantic analyses will have an important place in the study of online textual content on media and culture [95]. This study is not without its limitations. Aside from not offering the potential to conduct a deeper, critical discourse analysis [e.g. 36], which can be a highly useful addition to word embedding models [96, 97], an important limitation of our approach is that it explains little about the valence of words sharing a vector space. In other words, it does not tell us to what extent words are positively or negatively associated with each other or with positivity/negativity; in general, we only know that these discourses are unrelated to numerical evaluations provided by Pitchfork’s reviewers. This is a key inclusion that future research can provide, specifically qualitative, hermeneutic methodologies such as (critical) discourse analysis would offer more substantial empirical evidence in relation to the findings drawn from computation social science methods [96, 98]. A second limitation is that we cannot make causal inferences that explain the patterns we found. Previous research on associations between genres and gender/legitimacy is in line with our findings. Particularly, the patterns we found for metal, rap/hip-hop and jazz and the general trend towards legitimacy and femininity are in line with increasing egalitarian and progressive attitudes within the popular music media landscape. Nevertheless, this article demonstrates that methods from computational science carry substantial value for social science studies by capturing broad discursive patterns over a long period of time, which are difficult to capture by other means [99].
Data availability statement
The dataset ranging from 1999 to 2019 analysed in the current study is available in the Components repository, from https://components.one/datasets/pitchfork-reviews-dataset. The additional years 2019–2021 were provided by Components on personal request.
Notes
A figure depicting the number of reviews over time differentiated by genre can be found in the online supplement to this article.
CMDist is now subsumed under the text2map package [84].
We also fitted other models. For example, is the association between the legitimacy discourse and the gender discourse stable over time? Or does it differ between genres? These models did not substantially improve model fit and do not help in getting at a general understanding of the way both types of discourses relate to each other.
Available on request.
For some combinations of year and genre, there are very few observations. For example, in the first years, there are barely any reviews in the ‘global’ genre. Combinations of year and genre that have less than ten observations are not included in the visualization.
References
Janssen, S. (1999). Art journalism and cultural change: The coverage of the arts in Dutch newspapers 1965–1990. Poetics, 26(5–6), 329–348.
Barna, E. (2020). The relentless rise of the poptimist omnivore: Taste, symbolic power, and the digitization of the music industries. In T. Tofalvy & E. Barna (Eds.), Popular music, technology, and the changing media ecosystem (pp. 79–95). Palgrave Macmillan.
Verboord, M. (2014). The impact of peer-produced criticism on cultural evaluation: A multilevel analysis of discourse employment in online and offline film reviews. New Media & Society, 16(6), 921–940.
Debenedetti, S. (2006). The role of media critics in the cultural industries. International Journal of Arts Management, 8(3), 30–42.
Roose, H., Roose, W., & Daenekindt, S. (2018). Trends in contemporary art discourse: Using topic models to analyze 25 years of professional art criticism. Cultural Sociology, 12(3), 303–324.
Janssen, S., Kuipers, G., & Verboord, M. (2008). Cultural globalization and arts journalism: The international orientation of arts and culture coverage in Dutch, French, German, and US newspapers, 1955 to 2005. American Sociological Review, 73(5), 719–740.
Purhonen, S., Heikkilä, R., Karademir Hazir, I., Lauronen, T., Fernández Rodríguez, C. J., & Gronow, J. (2018). Enter culture, exit arts?: The transformation of cultural hierarchies in European newspaper culture sections, 1960–2010. Routledge.
Light, R., & Odden, C. (2017). Managing the boundaries of taste: Culture, valuation, and computational social science. Social Forces, 96(2), 877–908.
Binder, A. (1993). Constructing racial rhetoric: Media depictions of harm in heavy metal and rap music. American Sociological Review, 58(6), 753–767.
Van Venrooij, A., & Schmutz, V. (2010). The evaluation of popular music in the United States, Germany and the Netherlands: A comparison of the use of high art and popular aesthetic criteria. Cultural Sociology, 4(3), 395–421.
Peterson, R. A., & Kern, R. M. (1996). Changing highbrow taste: From snob to omnivore. American Sociological Review, 61(5), 900–907.
Van Eijck, K. (2000). Richard A. Peterson and the culture of consumption. Poetics, 28(2–3), 207–224.
Webster, J. (2019). Music on-demand: A commentary on the changing relationship between music taste, consumption and class in the streaming age. Big Data & Society, 6(2), 1–5.
Webster, J. (2020). Taste in the platform age: Music streaming services and new forms of class distinction. Information, Communication & Society, 23(13), 1909–1924.
Berkers, P., Verboord, M., & Weij, F. (2016). “These critics (still) don’t write enough about women artists”: Gender inequality in the newspaper coverage of arts and culture in France, Germany, the Netherlands, and the United States, 1955–2005. Gender & Society, 30(3), 515–539.
Johnson, D. (2011). Devaluing and revaluing seriality: The gendered discourses of media franchising. Media, Culture & Society, 33(7), 1077–1093.
Schmutz, V. (2009). Social and symbolic boundaries in newspaper coverage of music, 1955–2005: Gender and genre in the US, France, Germany, and the Netherlands. Poetics, 37(4), 298–314.
Lena, J. C., & Peterson, R. A. (2008). Classification as culture: Types and trajectories of music genres. American Sociological Review, 73(5), 697–718.
DiMaggio, P. (1982). Cultural entrepreneurship in nineteenth-century Boston: The creation of an organizational base for high culture in America. Media, Culture & Society, 4(1), 33–50.
Hebert, D. G. (2011). Originality and institutionalization: Factors engendering resistance to popular music pedagogy in the USA. Music Education Research International, 5, 12–21.
Zuidervaart, L. (2010). Art in public: Politics, economics, and a democratic culture. Cambridge University Press.
Kramer, M. J. (2012). Rocktimism?: Pop music writing in the age of rock criticism. Journal of Popular Music Studies, 24(4), 590–600.
Wilson, C. (2012). All that is solid melts into Schmaltz: Poptimism vs. the guilty displeasure. In E. Weisbard (Ed.), Pop when the world falls apart (pp. 299–312). Duke University Press.
Wilson, C. (2014). Let’s talk about love: Why other people have such bad taste. Bloomsbury Publishing.
Jarman-Ivens, F. (2007). Oh boy!: Masculinities and popular music. Routledge.
Hansen, K. A. (2021). Pop masculinities: The politics of gender in twenty-first century popular music. Oxford University Press.
Hansen, K. A. (2021). Staging a ‘real’ masculinity in a ‘fake’ world: Creativity, (in) authenticity, and the gendering of musical labour. Cultural Studies, 36(5), 713–731.
Miller, D. L. (2014). Symbolic capital and gender: Evidence from two cultural fields. Cultural Sociology, 8(4), 462–482.
Schmutz, V., & Faupel, A. (2010). Gender and cultural consecration in popular music. Social Forces, 89(2), 685–707.
Baumann, S. (2007). A general theory of artistic legitimation: How art worlds are like social movements. Poetics, 35(1), 47–65.
Schwenger, P. (2014). Phallic critiques: Masculinity and twentieth-century literature. Routledge.
Bielby, D. D., & Bielby, W. T. (1996). Women and men in film: Gender inequality among writers in a culture industry. Gender & Society, 10(3), 248–270.
Ferree, M. M., & Merrill, D. A. (2000). Hot movements, cold cognition: Thinking about social movements in gendered frames. Contemporary Sociology, 29(3), 454–462.
Fairclough, N. (1995). Media discourse. Edward Arnold.
Van Dijk, T. (1988). News analysis: Case studies of international and national news in the press. Lawrence Erlbaum.
Richardson, J. E. (2007). Analysing newspapers: An approach from critical discourse analysis. Palgrave Macmillan.
Allen, M. P., & Lincoln, A. E. (2004). Critical discourse and the cultural consecration of American films. Social Forces, 82(3), 871–894.
Baumann, S. (2001). Intellectualization and art world development: Film in the United States. American Sociological Review, 66(3), 404–426.
Hicks, A., & Petrova, V. (2006). Auteur discourse and the cultural consecration of American films. Poetics, 34(3), 180–203.
Regev, M. (1994). Producing artistic value: The case of rock music. The Sociological Quarterly, 35(1), 85–102.
Becker, H. S. (1982). Art worlds. University of California Press.
Bourdieu, P. (1993). The field of cultural production: Essays on art and literature. Polity Press.
Lopes, P. C. (2002). The rise of a jazz art world. Cambridge University Press.
Moeller, R. A. (2016). A question of legitimacy: Graphic novel reading as “real” reading. Journal of Adolescent & Adult Literacy, 59(6), 709–717.
Pizzino, C. (2016). Arresting development: Comics at the boundaries of literature. University of Texas Press.
Parker, F., Whitson, J. R., & Simon, B. (2018). Megabooth: The cultural intermediation of indie games. New Media & Society, 20(5), 1953–1972.
Styhre, A., Szczepanska, A. M., & Remneland-Wikhamn, B. (2018). Consecrating video games as cultural artifacts: Intellectual legitimation as a source of industry renewal. Scandinavian Journal of Management, 34(1), 22–28.
Bourdieu, P. (1984). Distinction: A social critique of the judgement of taste. Routledge.
Arnold, M. (2020). Culture and anarchy. In J. Z. Muller (Ed.), Conservatism (pp. 167–186). Princeton University Press.
Adorno, T., & Horkheimer, M. (1986). Dialectic of enlightenment. Verso.
Austerlitz, S. (2014). The pernicious rise of poptimism. The New York Times, Retrieved June 10, 2022, from https://www.nytimes.com/2014/04/06/magazine/the-pernicious-rise-of-poptimism.html
Berkers, P., & Schaap, J. (2018). Gender inequality in metal music production. Emerald Group Publishing.
Christenson, P. G., & Peterson, J. B. (1988). Genre and gender in the structure of music preferences. Communication Research, 15(3), 282–301.
Christenson, P. G., & Roberts, D. F. (1998). It’s not only rock & roll: Popular music in the lives of adolescents. Journal of Communication, 49(4), 212–229.
Colley, A. (2008). Young people’s musical taste: Relationship with gender and gender-related traits. Journal of Applied Social Psychology, 38(8), 2039–2055.
Kearney, M. C. (2017). Gender and rock. Oxford University Press.
Van Wel, F. V., Maarsingh, W., Bogt, T. T., & Raaijmakers, Q. (2008). Youth cultural styles: From snob to pop? Young, 16(3), 325–340.
Frith, S., & McRobbie, A. (1990). Rock and sexuality. In S. Frith & A. Goodwin (Eds.), On record: Rock, pop and the written word (pp. 371–389). Routledge.
Schippers, M. (2002). Rockin’out of the box: Gender maneuvering in alternative hard rock. Rutgers University Press.
Berkers, P., & Eeckelaer, M. (2014). Rock and roll or rock and fall? Gendered framing of the rock and roll lifestyles of Amy Winehouse and Pete Doherty in British broadsheets. Journal of Gender Studies, 23(1), 3–17.
Leonard, M. (2007). Gender in the music industry: Rock, discourse and girl power. Ashgate Publishing.
Miller, D. L. (2016). Gender, field, and habitus: How gendered dispositions reproduce fields of cultural production. Sociological Forum, 31(2), 330–353.
Davies, H. (2001). All rock and roll is homosocial: The representation of women in the British rock music press. Popular Music, 20(3), 301–319.
Johnson-Grau, B. (2002). Sweet nothings: Presentation of women musicians in pop journalism. In S. Jones (Ed.), Pop music and the press (pp. 202–218). Temple University Press.
Coates, N. (1997). (R)evolution now? Rock and the political potential of gender. In S. Whiteley (Ed.), Sexing the groove: Popular music and gender (pp. 50–64). Routledge.
Schaap, J., & Berkers, P. (2020). “Maybe it’s… skin colour?” How race-ethnicity and gender function in consumers’ formation of classification styles of cultural content. Consumption Markets & Culture, 23(6), 599–615.
Bannister, M. (2017). White boys, white noise: Masculinities and 1980s indie guitar rock. Routledge.
Fregault, J., Santoso, K., Lu, A., & Shah, I. (2020). Visualizing trends and patterns in Pitchfork reviews. DataRES at UCLA. Retrieved May 30, 2022, from https://ucladatares.medium.com/visualizing-trends-and-patterns-in-pitchfork-reviews-cad1a130869b
Henderson, S. (2008). Canadian content regulations and the formation of a national scene. Popular Music, 27(2), 307–315.
Similarweb (n.d.). Pitchfork traffic analytics and market share. Retrieved May 30, 2022, from https://www.similarweb.com/website/pitchfork.com/#overview
Mikolov, T., Chen, K., Corrado, G., & Dean, J. (2013). Efficient estimation of word representations in vector space. arXiv preprint arXiv:1301.3781.
Rodman, E. (2020). A timely intervention: Tracking the changing meanings of political concepts with word vectors. Political Analysis, 28(1), 87–111.
Kozlowski, A. C., Taddy, M., & Evans, J. A. (2019). The geometry of culture: Analyzing the meanings of class through word embeddings. American Sociological Review, 84(5), 905–949.
Nelson, L. K. (2021). Leveraging the alignment between machine learning and intersectionality: Using word embeddings to measure intersectional experiences of the nineteenth century US South. Poetics, 88, 101539.
Stoltz, D. S., & Taylor, M. A. (2021). Cultural cartography with word embeddings. Poetics, 88, 101567.
Stoltz, D. S., & Taylor, M. A. (2019). Concept Mover’s Distance: Measuring concept engagement via word embeddings in texts. Journal of Computational Social Science, 2(2), 293–313.
Taylor, M. A., & Stoltz, D. S. (2020). Integrating semantic directions with Concept Mover’s Distance to measure binary concept engagement. Journal of Computational Social Science, 4, 231–242.
Bojanowski, P., Grave, E., Joulin, A., & Mikolov, T. (2017). Enriching word vectors with subword information. Transactions of the Association for Computational Linguistics, 5, 135–146.
Mikolov, T., Grave, E., Bojanowski, P., Puhrsch, C., & Joulin, A. (2017). Advances in pre-training distributed word representations. arXiv preprint arXiv:1712.09405.
Clawson, M. A. (1999). When women play the bass: Instrument specialization and gender interpretation in alternative rock music. Gender & Society, 13(2), 193–210.
Lena, J. C. (2012). Banding together: How communities create genres in popular music. Princeton University Press.
Components (n.d.). 20,783 Pitchfork Reviews. Retrieved June 1, 2021, from The dataset analysed in the current study is available in the Components repository, Components from https://components.one/datasets/pitchfork-reviews-dataset.
Stoltz, D. S., & Taylor, M. A. (2021). CMDist: Concept Mover’s Distance and supporting functions. R package version 0.6.0. Retrieved October 5, 2021, from https://github.com/dustin-stoltz/CMDist
Stoltz, D. S., & Taylor, M. A. (2022). text2map: R tools for text matrices. Journal of Open Source Software, 7(72), 3741.
Koreman, R. (2014). Legitimating local music: Volksmuziek, hip-hop/rap and dance music in Dutch elite newspapers. Cultural Sociology, 8(4), 501–519.
Van Venrooij, A. (2009). The aesthetic discourse space of popular music: 1985–86 and 2004–05. Poetics, 37(4), 315–332.
Hill, R. L. (2016). Gender, metal and the media: Women fans and the gendered experience of music. Palgrave MacMillan.
Lizardo, O., & Skiles, S. (2016). Cultural objects as prisms: Perceived audience composition of musical genres as a resource for symbolic exclusion. Socius: Sociological Research for a Dynamic World, 2, 1–17.
White, M. (2011). From Jim Crow to Jay-Z: Race, rap, and the performance of masculinity. University of Illinois Press.
Bryson, B. (1996). “Anything but heavy metal”: Symbolic exclusion and musical dislikes. American Sociological Review, 61(5), 884–899.
Daenekindt, S., & Roose, H. (2017). Ways of preferring: Distinction through the ‘what’ and the ‘how’ of cultural consumption. Journal of Consumer Culture, 17(1), 25–45.
Schaap, J., Van der Waal, J., & De Koster, W. (2021). Black rock, white rock: non-declarative culture and the racialization of cultural categories. Sociological Inquiry. https://doi.org/10.1111/soin.12461
Brekhus, W. H. (2015). Culture and cognition: Patterns in the social construction of reality. Polity Press.
Zerubavel, E. (1999). Social mindscapes: An invitation to cognitive sociology. Harvard University Press.
Negus, K. (2019). From creator to data: The post-record music industry and the digital conglomerates. Media, Culture & Society, 41(3), 367–384.
Nelson, L. K. (2020). Computational grounded theory: A methodological framework. Sociological Methods & Research, 49(1), 3–42.
Törnberg, P., & Uitermark, J. (2021). For a heterodox computational social science. Big Data & Society, 8(2), 20539517211047724.
Törnberg, P., & Uitermark, J. (2021). For a heterodox computational social science. Big Data & Society, 8(2), 1–13.
Pitchfork (n.d.). Pitchfork. Retrieved January 31, 2022, from https://pitchfork.com/
Acknowledgements
The authors would like to thank Michaël Berghman, Thomas Calkins, Koen van Eijck, Charlotte Peeters, Femke Vandenberg, Jef Vlegels, and four anonymous reviewers for valuable feedback on earlier versions of this article. In addition, we thank Andrew Thompson for providing the data for this article.
Funding
This research received no financial support from any specific funding body.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
On behalf of all authors, the corresponding author states that there is no conflict of interest.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Supplementary Information
Below is the link to the electronic supplementary material.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Daenekindt, S., Schaap, J. Using word embedding models to capture changing media discourses: a study on the role of legitimacy, gender and genre in 24,000 music reviews, 1999–2021. J Comput Soc Sc 5, 1615–1636 (2022). https://doi.org/10.1007/s42001-022-00182-8
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s42001-022-00182-8