Skip to main content

A mathematical approach to assess research diversity: operationalization and applicability in communication sciences, political science, and beyond

Scientometrics volume 125, pages 2299–2322 (2020)Cite this article

Abstract

With today’s research production and global dissemination, there is growing pressure to assess how academic fields foster diversity. Based on a mathematical problem/solve scheme, the aim of this study is twofold. First, the paper elaborates on how research diversity in scientific fields can be empirically gauged, proposing six working definitions. Second, drawing on these theoretical explanations, we introduce an original methodological protocol for research diversity evaluation. Third, the study puts this mathematical model to an empirical test by comparatively evaluating (1) communication research diversity in 2017, with respect to field’s diversity in 1997, and (2) communication research and political science diversity in 2017. Our results indicate that, contrasted to pattern diversity, communication research in 2017 is not a diverse field. However, throughout the years (1997–2017), there is a statistically significant improvement. Finally, the cross-comparison examination between political and communication sciences reveals the latter to be significantly more diverse.

This is a preview of subscription content, access via your institution.

Access options

Buy single article

Instant access to the full article PDF.

USD 39.95

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Fig. 1
Fig. 2

Notes

  1. \(X_{1} =\) First author affiliation; \(X_{2} =\) First author gender; \(X_{3} =\) First author ethnicity; \(X_{4} =\) First author affiliation type; \(X_{5} =\) Type of authorship; \(X_{6} =\) Form of collaboration; \(X_{7} =\) Interdisciplinarity; \(X_{8} =\) Area of data collection; \(X_{9} =\) Methodologies; \(X_{10} =\) Research approach; \(X_{11} =\) Type of samples; \(X_{12} =\) Paradigms; \(X_{13} =\) Content area; \(X_{14} =\) Analytical focus; \(X_{15} =\) Theoretical framework.

  2. Remind that all bootstrap procedures are done case-wise in order to preserve the multivariate structure of the data, which may be of importance if variables are not independent.

  3. Note that all expected cell values are greater than 5, hence no Yates correction is needed. For example, if we compute expected cell values in the worst case, which are those corresponding to variable “area of data collection” with k = 13 categories, we have that for a sample size of n = 283, they are n·1/k = 21.77.

  4. Note that, all expected cell values are greater than 5, hence no Yates correction is needed. For example, if we compute expected cell values in the worst case, which are those corresponding to variable “area of data collection” with k = 13 categories, we have that for a sample size of n = 263, they are n·1/k = 20.23.

References

  • Agresti, A., & Agresti, B. F. (1978). Statistical analysis of qualitative variation. In K. F. Schussler (Ed.), Social methodology (Vol. 9, pp. 204–237). New York: Wiley.

    Google Scholar 

  • Bhattacharyya, A. (1943). On a measure of divergence between two statistical populations defined by their probability distributions. Bulletin of the Calcutta Mathematical Society, 35, 99–109.

    MathSciNet  MATH  Google Scholar 

  • Bone, F., Hopkins, M. M., Ráfols, I., Molas-Gallart, J., Tang, P., Davey, G., & Carr, A. M. (2019). DARE to be different? Applying diversity indicators to the evaluation of collaborative research projects. Science Policy Research Unit—SPRU working paper series 2019–09, University of Sussex, UK.

  • Borgman, C. L. (1989). Bibliometrics and scholarly communication: Editor’s introduction. Communication Research, 16(5), 583–599.

    Article  Google Scholar 

  • Boschma, R. (2005). Proximity and innovation: A critical assessment. Regional Studies, 39(1), 61–74.

    Article  Google Scholar 

  • Bunz, U. (2005). Publish or perish: A limited author analysis of ICA and NCA journals. Journal of Communication, 55(4), 703–720. https://doi.org/10.1111/j.1460-2466.2005.tb03018.x.

    Article  Google Scholar 

  • Chakravartty, P., Kuo, R., Grubbs, V., & McIlwain, C. (2018). #CommunicationSoWhite. Journal of Communication, 68(2), 254–266. https://doi.org/10.1093/joc/jqy003.

    Article  Google Scholar 

  • Cohen, J. (1960). A coefficient of agreement for nominal scales. Educational and Psychological Measurement, 20(1), 37–46.

    Article  Google Scholar 

  • Curran, J., & Park, M. (2000). De-Westernizing media studies. London: Routledge.

    Google Scholar 

  • Demeter, M. (2018). Changing center and stagnant periphery in communication and media studies: National diversity of major international journals in the field of communication from 2013 to 2017. International Journal of Communication, 12, 29.

    Google Scholar 

  • Dhanani, A., & Jones, M. J. (2017). Editorial boards of accounting journals: Gender diversity and internationalisation. Accounting, Auditing & Accountability Journal, 30(5), 1008–1040. https://doi.org/10.1108/AAAJ-08-2014-1785.

    Article  Google Scholar 

  • Everitt, B. S., & Skrondal, A. (2010). The Cambridge dictionary of statistics (4th ed.). New York: Cambridge University Press.

    Book  Google Scholar 

  • Feeley, T. H. (2008). A bibliometric analysis of communication journals from 2002 to 2005. Human Communication Researh, 34(3), 505–520. https://doi.org/10.1111/j.1468-2958.2008.00330.x.

    Article  Google Scholar 

  • Freelon, D. (2013). Co-citation map of 9 comm journals, 2003–2013. Retrieved May 5, 2020, from http://dfreelon.org/2013/09/05/co-citation-map-of-9-comm-journals-2003-2013/.

  • Funkhouser, E. T. (1996). The evaluative use of citation analysis for communication journals. Human Communication Research, 22(4), 563–574. https://doi.org/10.1111/j.1468-2958.1996.tb00379.x.

    Article  Google Scholar 

  • Ganter, S. A., & Ortega, F. (2019). The invisibility of Latin American Scholarship in European media and communication studies: Challenges and opportunities of de-westernization and academic cosmopolitanism. International Journal of Communication, 13, 68–91.

    Google Scholar 

  • Gil de Zuniga, H., & Diehl, T. (2017). Citizenship, social media, and big data: Current and future research in the social sciences. Social Science Computer Review, 35(1), 3–9.

    Article  Google Scholar 

  • Gini, C. (1912). Variabiliti e Mutabiliti. Studi Economicoaguridici della facotta di Giurisprudenza dell. Cagliari: Universite di Cagliari III, Parte II.

    Google Scholar 

  • Goyanes, M. (2020). Editorial boards in communication sciences journals: Plurality or standardization? International Communication Gazette, 82(4), 342–364.

    Article  Google Scholar 

  • Goyanes, M., & Demeter, M. (2020). How the geographic diversity of editorial boards affects what is published in JCR-ranked communication journals. Journalism & Mass Communication Quarterly. https://doi.org/10.1177/1077699020904169.

  • Griffin, D. J., Bolkan, S., Holmgren, J. L., & Tutzauer, F. (2016). Central journals and authors in communication using a publication network. Scientometrics, 106(1), 91–104. https://doi.org/10.1007/s11192-015-1774-4.

    Article  Google Scholar 

  • Guenther, L., & Joubert, M. (2017). Science communication as a field of research: Identifying trends, challenges and gaps by analysing research papers. Journal of Science Communication, 16(2), 1–19. https://doi.org/10.22323/2.16020202.

    Article  Google Scholar 

  • Günther, E., & Domahidi, E. (2017). What communication scholars write about: An analysis of 80 years of research in high-impact journals. International Journal of Communication, 11, 3051–3071.

    Google Scholar 

  • Hendrix, K. G., Mazer, J. P., & Hess, J. A. (2016). Forum: Diversity and scholarship on instructional communication. Communication Education, 65(1), 105–127.

    Article  Google Scholar 

  • Hirschman, A. O. (2018). National power and the structure of foreign trade. Berkeley: University of California Press.

    Google Scholar 

  • Keating, D. M., Richards, A. S., Palomares, N. A., Banas, J. A., Joyce, N., & Rains, S. A. (2019). Titling practices and their implications in communication research 1970–2010: Cutesy cues carry citation consequences. Communication Research. https://doi.org/10.1177/0093650219887025.

    Article  Google Scholar 

  • Knobloch-Westerwick, S., & Glynn, C. J. (2013). The Matilda effect—Role congruity effects on scholarly communication: A citation analysis of communication research and journal of communication articles. Communication Research, 40(1), 3–26. https://doi.org/10.1177/0093650211418339.

    Article  Google Scholar 

  • Landis, J. R., & Koch, G. G. (1977). The measurement of observer agreement for categorical data. Biometrics, 33(1), 159–174.

    Article  Google Scholar 

  • Lauf, E. (2005). National diversity of major international journals in the field of communication. Journal of Communication, 55(1), 139–151. https://doi.org/10.1111/j.1460-2466.2005.tb02663.x.

    Article  Google Scholar 

  • Leeds-Hurwitz, W. (2019). Moving (slowly) toward understanding knowledge as a global commons. Journal of Multicultural Discourses. https://doi.org/10.1080/17447143.2019.1695806.

    Article  Google Scholar 

  • Leydesdorff, L., & Probst, C. (2009). The delineation of an interdisciplinary specialty in terms of a journal set: The case of communication studies. Journal of the American Society for Information Science and Technology, 60(8), 1709–1718.

    Article  Google Scholar 

  • Leydesdorff, L., & Rafols, I. (2010). Indicators of the interdisciplinarity of journals: Diversity, centrality, and citations. Journal of Informetrics, 5(1), 87–100.

    Article  Google Scholar 

  • Leydesdorff, L., Wagner, C. S., & Bornmann, L. (2019). Interdisciplinarity as diversity in citation patterns among journals: Rao–Stirling diversity, relative variety, and the Gini coefficient. Journal of Informetrics, 13(1), 255–269.

    Article  Google Scholar 

  • Livingstone, S. (2007). Internationalizing media and communication studies: Reflections on the International Communication Association. Global Media and Communication, 3(3), 273–288.

    Article  MathSciNet  Google Scholar 

  • Luthra, R. (2015). Transforming global communication research with a view to the margins. Communication Research and Practice, 1(3), 251–257. https://doi.org/10.1080/22041451.2015.1079156.

    Article  Google Scholar 

  • Magurran, A. E. (1988). Ecological diversity and its measurement. Princeton, NJ: Princeton University Press.

    Book  Google Scholar 

  • Metz, I., Harzing, A. W., & Zyphur, M. J. (2016). Of journal editors and editorial boards: who are the trailblazers in increasing editorial board gender equality? British Journal of Management, 27(4), 712–726. https://doi.org/10.1111/1467-8551.12133.

    Article  Google Scholar 

  • Nikulin, M. S. (1994). Hellinger distance. In Encyclopedia of mathematics. Retrieved May 5, 2020, from https://www.encyclopediaofmath.org/index.php/Hellinger_distance.

  • Paisley, W. (1989). Bibliometrics, scholarly communication, and communication research. Communication Research, 16(5), 701–717. https://doi.org/10.1177/009365089016005010.

    Article  Google Scholar 

  • Park, H., & Leydesdorff, L. (2009). Knowledge linkage structures in communication studies using citation analysis among communication journals. Scientometrics, 81(1), 157–175. https://doi.org/10.1007/s11192-009-2119-y.

    Article  Google Scholar 

  • Ràfols, I. (2014). Knowledge integration and diffusion: Measures and mapping of diversity and coherence. In Y. Ding, R. Rousseau, & D. Wolfram (Eds.), Measuring scholarly impact (pp. 169–190). Cham: Springer.

    Google Scholar 

  • Rafols, I., & Meyer, M. (2010). Diversity and network coherence as indicators of interdisciplinarity: Case studies in bionanoscience. Scientometrics, 82(2), 263–287.

    Article  Google Scholar 

  • Rao, C. R. (1948). The utilization of multiple measurements in problems of biological classification. Journal of the Royal Statistical Society, B, 13, 159–193.

    MathSciNet  MATH  Google Scholar 

  • Rao, C. R. (1982a). Diversity and dissimilarity coefficients: A unified approach. Theoretical Population Biology, 21(1), 24–43.

    Article  MathSciNet  Google Scholar 

  • Rao, C. R. (1982b). Diversity: Its measurement, decomposition, apportionment and analysis. Sankhya: The Indian Journal of Statistics, Series A, 44(1), 1–22.

    MathSciNet  MATH  Google Scholar 

  • Reeves, B., & Borgman, C. L. (1983). A bibliometric evaluation of core journals in communication research. Human Communication Research, 10(1), 119–136. https://doi.org/10.1111/j.1468-2958.1983.tb00007.x.

    Article  Google Scholar 

  • Rice, R. E., Borgman, C. L., & Reeves, B. (1988). Citation networks of communication journals, 1977–1985 cliques and positions, citations made and citations received. Human Communication Research, 15(2), 256–283. https://doi.org/10.1111/j.1468-2958.1988.tb00184.x.

    Article  Google Scholar 

  • Rogers, E. M. (1999). Anatomy of the two subdisciplines of communication study. Human Communication Research, 25(4), 618–631. https://doi.org/10.1111/j.1468-2958.1999.tb00465.x.

    Article  Google Scholar 

  • Rousseau, R. (2019). Correspondence. On the Leydesdorff–Wagner–Bornmann proposal for diversity measurements. Journal of Informetrics, 13, 906–907.

    Article  Google Scholar 

  • Shannon, C. E. (1948). A mathematical theory of communication. Bell System Technical Journal, 27(3), 379–423. https://doi.org/10.1002/j.1538-7305.1948.tb01338.x.

    Article  MathSciNet  MATH  Google Scholar 

  • Smith, E. O. (2000). Strength in the technical communication journals and diversity in the serials cited. Journal of Business and Technical Communication, 14(2), 131–184. https://doi.org/10.1177/105065190001400201.

    Article  Google Scholar 

  • So, C. Y. (1988). Citation patterns of core communication journals: An assessment of the developmental status of communication. Human Communication Research, 15(2), 236–255. https://doi.org/10.1111/j.1468-2958.1988.tb00183.x.

    Article  Google Scholar 

  • Sokal, R. R., & Sneath, P. H. A. (1963). Principles of numerical taxonomy. San Francisco: Freeman.

    MATH  Google Scholar 

  • Stephan, P. E., & Levin, S. G. (1991). Inequality in scientific performance: Adjustment for attribution and journal impact. Social Studies of Science, 21(2), 351–368. https://doi.org/10.1177/030631291021002007.

    Article  Google Scholar 

  • Stirling, A. (2007). A general framework for analyzing diversity in science, technology and society. Journal of the Royal Society, Interface, 4(15), 707–719.

    Article  Google Scholar 

  • Toth, J. (2018). “U.S. journals can afford to remain regional, but we can not.” Author distribution-based internationality of Eastern European communication journals. KOME—An International Journal of Pure Communication Inquiry, 6(2), 1–15. https://doi.org/10.17646/KOME.2018.21.

    Article  Google Scholar 

  • Waisbord, S. (2016). Communication studies without frontiers? Translation and cosmopolitanism across academic cultures. International Journal of Communication, 10(2016), 868–886.

    Google Scholar 

  • Waisbord, S. (2019). Communication. A post-discipline. London: Polity Press.

    Google Scholar 

  • Waisbord, S., & Mellado, C. (2014). De-westernizing communication studies: A reassessment. Communication Theory, 24(4), 361–372. https://doi.org/10.1111/comt.12044.

    Article  Google Scholar 

  • Walter, N., Cody, M. J., & Ball-Rokeach, S. J. (2018). The ebb and flow of communication research: Seven decades of publication trends and research priorities. Journal of Communication, 68(2), 424–440. https://doi.org/10.1093/joc/jqx015.

    Article  Google Scholar 

  • Wasserman, H. (2018). Power, meaning and geopolitics: Ethics as an entry point for global communication studies. Journal of Communication, 68(2), 441–451. https://doi.org/10.1093/joc/jqy001.

    Article  Google Scholar 

  • Willems, W. (2014). Provincializing hegemonic histories of media and communication studies: Toward a genealogy of epistemic resistance in Africa. Communication Theory, 24(4), 415–434. https://doi.org/10.1111/comt.12043.

    Article  Google Scholar 

  • Youk, S., & Park, H. S. (2019). Where and what do they publish? Editors’ and editorial board members’ affiliated institutions and the citation counts of their endogenous publications in the field of communication. Scientometrics, 120(3), 1237–1260. https://doi.org/10.1007/s11192-019-03169-x.

    Article  Google Scholar 

  • Zhang, L., Glänzel, W., & Liang, L. M. (2009). Tracing the role of individual journals in a cross-citation network based on different indicators. Scientometrics, 81(3), 821–838.

    Article  Google Scholar 

  • Zhang, L., Janssens, F., Liang, L. M., & Glänzel, W. (2010). Journal crosscitation analysis for validation and improvement of journal-based subject classification in bibliometric research. Scientometrics, 82(3), 687–706.

    Article  Google Scholar 

  • Zhang, L., Rousseau, R., & Glänzel, W. (2016). Diversity of references as an indicator for interdisciplinarity of journals: Taking similarity between subject fields into account. Journal of the Association for Information Science and Technology, 67(5), 1257–1265.

    Article  Google Scholar 

  • Zhu, Y., & Fu, K. W. (2019). The Relationship between interdisciplinarity and journal impact factor in the field of communication during 1997–2016. Journal of Communication, 69(3), 273–297. https://doi.org/10.1093/joc/jqz012.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Communication, Carlos III University, C/Madrid 133, Madrid, Spain

    Manuel Goyanes

  2. Democracy Research Unit (DRU), Political Science, University of Salamanca, Salamanca, Spain

    Manuel Goyanes & Homero Gil de Zúñiga

  3. Department of Social Communication, National University of Public Service, Budapest, Hungary

    Márton Demeter

  4. Department of Statistics, Universidad Carlos III de Madrid, Madrid, Spain

    Aurea Grané & Irene Albarrán-Lozano

  5. Department of Film Production and Media Studies, Pennsylvania State University, State College, USA

    Homero Gil de Zúñiga

  6. Facultad de Comunicación y Letras, Universidad Diego Portales, Santiago, Chile

    Homero Gil de Zúñiga

Authors
  1. Manuel Goyanes
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Márton Demeter
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Aurea Grané
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Irene Albarrán-Lozano
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Homero Gil de Zúñiga
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Manuel Goyanes.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (DOCX 248 kb)

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Goyanes, M., Demeter, M., Grané, A. et al. A mathematical approach to assess research diversity: operationalization and applicability in communication sciences, political science, and beyond. Scientometrics 125, 2299–2322 (2020). https://doi.org/10.1007/s11192-020-03680-6

Download citation

  • Received:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11192-020-03680-6

Keywords

  • Research diversity
  • Diversity
  • Communication science
  • Political science
  • Diversity gaps