Skip to main content

Transforming Academic Journal Assessment from “Quality” to “Impact”: A Case Study of the SDG Impact Intensity Academic Journal Rating Artificial Intelligence System

  • Chapter
  • First Online:
The Future of Responsible Management Education

Part of the book series: Humanism in Business Series ((HUBUS))

Abstract

This chapter details how artificial intelligence (AI) can be used—and is very much needed—to reinvent academic research evaluation in terms of quality and impact. First, we review current literature on academic research assessment examining two overarching questions: What normative standards are employed to determine the quality of academic research? What explicit methodological rationales, unstated assumptions, and academic cultural dynamics determine the definition of quality? Second, we offer a literature review and alternative model for assessing the quality of academic research in terms of impact—that is, does academic research make a positive difference in the material conditions of humanity and sustain the natural environment, and how does academic research praxis transform “words into worlds”? Third, we examine how impact is being adopted by the publishing industry in a paradigm shift from limited notions of “quality” to the broader consequences of societal and sustainable “impact.” We critique how the Journal Impact Factor, as a gold standard of academic publication value, has eclipsed other possibilities for assessing real-world impacts of academic outputs: quantification without qualification. Fourth, we present a case study applying an AI rating system to ascertain degrees of impact for academic journals—SDG Impact Intensity. The AI system utilizes the United Nations 17 Sustainable Development Goals (SDGs) criteria as normative standards to rate the impact of academic journals. We discuss the implications of such a system and SDG-focused research on business schools and scholarly outputs for academia. We critically consider the challenges of employing our algorithmic technique as a digital transformation technology for responsible management education, and ultimately to serve the Common Good and environmental sustainability.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Subscribe and save

Springer+ Basic
EUR 32.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 149.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 199.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 199.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

Notes

  1. 1.

    This publication and research effort are made possible through the financial support of the Johnson & Johnson Foundation, the Haub School of Business at Saint Joseph’s University, and Cabells. We especially thank Cabells for their full support and extensive collaboration on the development and deployment of this research. We gratefully acknowledge the faculty and staff of the Haub School SDG Dashboard Team for their innovative and invaluable contributions.

  2. 2.

    “UN SDG related issues for management” is a subgateway of the more encompassing Responsible Management Gateway.

  3. 3.

    Some of these journals would not be categorically considered business and management journals by traditional standards. However, these business-related journals are necessary to include because of the diversity and scope of the SDGs—achieving the SDGs is an interdisciplinary undertaking, so business scholars should and do frequently publish scholarship in these nontraditional journals. Outside the scope of this chapter, this move to be more inclusive of what even “counts” as a business- and management-related journal is also part of a paradigmatic upheaval of how we conceive of “proper” business and management research and journals.

  4. 4.

    An important note disentangling differing ideas of impact. For Garfield and the impact factor phenomena that followed, impact is construed on the basis of quantitative counting of citations. The type of impact discussed in this chapter considers social and environmental impact, constructed from a different qualitative, normative set of operating principles sourced from the SDGs.

  5. 5.

    Crawford (2020, p. 496) provides a helpful distinction of how adding considerations of impact is not “supplanting” traditional metrics, but “supplementing” them.

  6. 6.

    Retraction Watch (https://retractionwatch.com) provides a curated and constantly updated collection of retractions and some ethics-related academic publishing violations.

References

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to David Steingard .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Steingard, D., Linacre, S. (2023). Transforming Academic Journal Assessment from “Quality” to “Impact”: A Case Study of the SDG Impact Intensity Academic Journal Rating Artificial Intelligence System. In: Hauser, C., Amann, W. (eds) The Future of Responsible Management Education. Humanism in Business Series. Palgrave Macmillan, Cham. https://doi.org/10.1007/978-3-031-15632-8_17

Download citation

Publish with us

Policies and ethics