Why the Community Should Care About Technology-Centric Journal Rankings

Recently, a talented junior colleague from Western Europe was highly interested in a posted job opening for a full professorship at a business school in a European city. The description of the position had a strong technical IS focus: software development, enterprise systems, and business process management – everything that we claim at the core of our discipline. And it was exactly the professional background of the assistant professor. Since the young colleague did not know the hiring organization, s/he contacted a colleague from the search committee. Requirements and capabilities seemed to match perfectly and looked potentially beneficial for both parties: the job candidate and the business school.

After constructive and open conversations with the colleague from the search committee, the assistant professor decided not to apply for this position. There was a fundamental misalignment between the searching organization and the candidate regarding future publication expectations. The designated employer, an ambitious business school on the rise, expected the candidate to publish in the University of Texas at Dallas (UTD) list of leading business journals (Dallas 2017). From an IS perspective, two specific journals are listed in this ranking: Information Systems Research (ISR) and Management Information Systems Quarterly (MISQ). Furthermore, Management Science (MS) and Production and Operations Management (POM) have dedicated IS departments which can be considered as additional targets.

These journals publish very high-quality research papers which address fundamental managerial problems. In particular MISQ, and to a significant extent ISR, focuses on empirically grounded work which is guided by socio-economic theories. This was at the opposite publication spectrum of the young talent. S/he preferred the design and implementation of novel technological artifacts with a strong analytical flavor. It is rather counter-intuitive to build a future on a research paradigm that does not match one’s own research interests and capabilities. Thus, the candidate decided not to hand in an application for this position.

The decision was certainly understandable but it should not be overlooked that the outcome was anything but optimal: (a) The business school offering the position was not able to attract a renowned talent and (b) the talent realized that vacancies in business schools may provide limited perspectives. As a consequence, students in the respective business school could end up being taught software engineering or business process management by a scholar who never wrote a line of code or who never mined a business process. At the same time, the IS job market becomes less attractive for young talents.

On the one hand, recent years have seen breath-taking developments in machine learning (e.g., the successes of deep learning), main-memory computing (e.g., main memory databases), and computational optimization (e.g., faster algorithms for mixed integer programming). Many sciences such as biology or computer vision are extremely agile in adopting these technologies, but such computational or algorithmic approaches might not fit journals such as MISQ. On the other hand, young academic talents with related interests cannot develop their capabilities since they do not get tenure. Such a lose–lose situation negatively affects the job market for young scholars with a technical focus and the respective employing institutions. This may lead to adverse selection processes which are a jeopardy for the entire community. More importantly, a narrow focus on a few journals steers our field into a narrow, mono-methodological direction. As a consequence, we might not be able to leverage the potential of new technologies in a way that nurtures our entire community. Writing about technology is something different than innovating with technology!

What can research organizations do to avoid such lose–lose situations? The vast majority of IS positions resides within business schools. The competition among them has significantly intensified. Thus, these schools are looking for metrics that  help them capture the quality of the research of their own faculty as well as the quality of research of prospective new faculty members. No matter whether a school uses the UTD 24, the FT50 (Financial Times 2016) or the VHB JOURQUAL (Frank et al. 2008) ranking, these ratings have managed to become de facto standards. Since the rankings are based on a finite set of journals, the research output becomes comparable between organizations and scholars. This, in turn, means that all IS scholars are pressured to publish in ISR or MISQ which largely excludes the technology or algorithmic side of our discipline. Although these rankings possess considerable imperfections, in particular, the neglect of the research impact, they are on the deans’ agenda since they allow for inter-organizational and inter-personal comparisons.

Very few business schools have extended their journal lists to information technology and computer science outlets with high impact. Based on this practice, an attempt was made to break up this disadvantageous practice. The respective colleague wanted to support the extension idea but his dean asked whether a well-accepted journal list existed that included such information technology and computer science centric outlets. Unfortunately, such a list exists for scarcely any computer science.

Do we learn from this unfortunate situation that imperfect, but well-established quality standards seem to be better than none at all?

Since research outputs on the technology side of our discipline have not been able to establish explicit and well-accepted standards, they put all colleagues in a vulnerable position that work at the core of our discipline: information technology. Furthermore, it inhibits the careers of our young talents which may suffer from an adverse selection process. This is particularly remarkable since a Memorandum of design-oriented IS research was drawn up at the beginning of this decade (Osterle et al. 2010). They claimed that build-orientation has the potential of becoming a differentiator compared to the Anglo-American Design Science notion, but it remains an empty rhetoric if quality standards stay undefined and unpursued. Strategic orientation looks different.

Only few academic associations have made an attempt to address this topic. For instance, after a controversial debate, the German-speaking community ratified a journal orientation list that included the technology side of IS research (WI-Orientierungslisten 2008). Though this effort was barely able to establish an additional standard, it has been successful in providing orientation for young IS scholars who have focused on the empirical side of IS research. The research activity of that respective community in international conferences and journals has significantly increased during the past decade. For instance, for the first time ever, the respective IS community was able to contribute the majority of ICIS submissions in Seoul 2017, leaving the US behind. But those colleagues that have been working on the technical side, however, have received less guidance.

Thus, this creeping issue is a vast challenge for all IS communities in Europe, Asia, Australia and North America. The Computing Research and Education Association of Australasia, CORE, offers a vivid example of what an extensive ranking can look like (CORE 2018). Better collective efforts are necessary to resolve this issue by creating extended, more balanced publication standards. The international IS community is well advised to develop a compact but high quality IS journal and conference list which captures major technologically oriented research outlets in terms of rigor and signaling. The German “Gesellschaft für Informatik” (GI) could play a catalyzing role in such an initiative. There are well-established outlets (conferences or journals) in software engineering, databases, distributed systems, algorithms, and other technical fields. If young scholars are able to contribute to these jointly specified targets, this could be regarded equally significant as a publication “hit” in a top marketing or strategy journal which is part of the UTD24 or FT50 list.

One of the contributions of the BISE community to business schools has always been the technical strength of its respective faculty. Nowadays, every business school dean has grasped the notion that graduates need to have a significant level of technical understanding in a digitalized world. If research in information systems neglects its technical elements and, thus, becomes a sub-domain of marketing, strategy, or operations research, this may become a precarious and almost irreversible legacy for the future. “Technology light” is unlikely to be a valuable and sustainable imperative for business schools and our community.

However, complementing existing journal lists by adding outlets focusing on technology may not go far enough, since it separates the technological from the socio-economic sub-communities in our discipline. In previous editorials, we have emphasized that technology is an outflow of economic and behavioral theory (Heinzl et al. 2015). Thus, an inherent concern is how to avoid a further fragmentation of our discipline.

A different and probably better option would be to nurture journals or conferences that combine or even integrate technology and socio-economic theory (Heinrich et al. 2011, pp. 83). Although the BISE journal has continued to develop extremely well since its international launch in 2009 and increased its ISI Impact Factor to 3.392 in 2017, there is still a lot of work ahead to reach this goal. Nevertheless, we are convinced that the pluralistic and integrative approach of our journal is a unique and solid foundation for moving up the ranking layers. With the support of all IS communities in different parts of the world – technological and managerial – this effort may provide even better orientation for young scholars, colleagues inside and outside our community as well as the deans of business schools and institutes of technology. Let us integrate these two different sides of the same coin and provide better orientation for all scholars inside and outside our community.

References

  1. CORE (2018) CORE rankings portal. http://www.core.edu.au/team. Accessed 2 Jan 2018

  2. Dallas UT (2017) The UTD top 100 business school research rankings. http://jindal.utdallas.edu/the-utd-top-100-business-school-research-rankings/list-of-journals. Accessed 29 Dec 2017

  3. Financial Times (2016) 50 journals used in FT research rank. https://www.ft.com/content/3405a512-5cbb-11e1-8f1f-00144feabdc0. Accessed 29 Dec 2017

  4. Heinrich LJ, Heinzl A, Riedl R (2011) Wirtschaftsinformatik – Einführung und Grundlegung. Springer, Heidelberg

    Google Scholar 

  5. Heinzl A, Winter R, Bichler M (2015) Internationalization of information systems research and teaching. Bus Inf Syst Eng 57(4):225–228

    Article  Google Scholar 

  6. Frank U, Heinzl A, Schoder D (2008) WI-Orientierungslisten – WI-Journalliste 2008 sowie WI-Liste der Konferenzen, Proceedings und Lecture Notes 2008. Wirtschaftsinformatik 50(2):155–163

    Google Scholar 

  7. Osterle H, Becker J, Frank U, Hess T, Karagiannis D, Krcmar H, Loos P, Mertens P, Oberweis A, Sinz EJ (2010) Memorandum on design-oriented information systems research. Eur J Inf Syst 20(1):7–10

    Article  Google Scholar 

  8. VHB (2017) VHB-JOURQUAL3. http://vhbonline.org/en/service/jourqual/vhb-jourqual-3/complete-list-of-the-journals/. Accessed 29 Dec 2017

Download references

Author information

Affiliations

Authors

Corresponding author

Correspondence to Prof. Dr. Armin Heinzl.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Heinzl, A., van der Aalst, W. & Bichler, M. Why the Community Should Care About Technology-Centric Journal Rankings. Bus Inf Syst Eng 60, 91–93 (2018). https://doi.org/10.1007/s12599-018-0519-3

Download citation