VHB-JOURQUAL2: Method, Results, and Implications of the German Academic Association for Business Research’s Journal Ranking

VHB(cid:11)JOURQUAL represents the official journal ranking of the German Academic Association for Business Research. Since its introduction in 2003, the ranking has become the most influential journal evaluation approach in German(cid:11)speaking countries, impacting several key managerial decisions of German, Aus(cid:11) trian, and Swiss business schools. This article reports the methodological approach of the ranking’s second edition. It also presents the main results and additional analyses on the validity of the rating and the un(cid:11) derlying decision processes of the respondents. Selected implications for researchers and higher(cid:11)education institutions are discussed.


Introduction
VHB JOURQUAL represents the official journal ranking of the German Academic Association for Business Research (Verband der Hochschullehrer für Betriebswirtschaftslehre -VHB). It rates and ranks international and German language academic journals which are considered relevant for German speaking business researchers based on the quality assessments of VHB members who comprise more than 90% of all professors and researchers with a Ph.D. in business administration at German, Aus trian, and Swiss universities. Since its initial publi cation in 2003 (Hennig Thurau, Walsh, and Schrader 2004), the ranking has become the most prominent business research journal ranking in German speaking countries. The results of an online survey of VHB members in 2007 (n = 489) give evidence that VHB JOUR QUAL is widely accepted and broadly used for evaluating the scientific performance of business scholars in Austria, Germany and the German speaking part of Switzerland (Schrader and Hennig Thurau 2007). 59 % of the respondents assessed VHB JOURQUAL as "good" or "very good", while only 9 % held a negative attitude toward the rank ing. VHB JOURQUAL is of special importance for the formal post doctoral assessment in German speaking countries -the so called "Habilitation," a traditional requirement for obtaining a full profes sorship -and the appointment of full professors; in each case, 54 % of the respondents judged the rank ing being of "high" or "very high" relevance for their institutions. Other areas for which VHB JOURQUAL is considered to have substantial rele vance include the hiring and evaluation of assistant professors ("Junior Professoren"), research per formance evaluations (with a possible impact on budget allocation), and the evaluation of the rapidly VHB JOURQUAL2: Method, Results, and Implications of the German Academic Asso ciation for Business Research's Journal Ranking growing number of cumulative doctoral disserta tions. VHB JOURQUAL is considered a major driver of the radical transformation of the German business administration community into a much more re search focused and internationally active academic discipline (Homburg 2008). In an environment which Simon (1993) described as a "black hole"with German speaking business scholars absorbing their international colleagues' findings, but giving nothing back to the international community -, VHB JOURQUAL offered for the first time a com parison of the scientific quality of articles published in German language journals with those published in international journals. As the ranking provided strong evidence that the quality of even the best German language journals is perceived to be sub stantially lower than the quality of leading interna tional journals, VHB JOURQUAL has since then become a strong motivating force for German schol ars to publish their best work internationally and enter "global competition". This article describes the methodology of VHB JOURQUAL and reports specific features and key results of the second edition of the ranking (VHB JOURQUAL2), which is based on a survey con ducted among VHB members in 2008. We also investigate the ranking's validity and provide con cluding remarks on the benefits and limitations of VHB JOURQUAL.

Background: Survey-based vs. citation based journal-rankings
There are two basic approaches for conducting aca demic journal rankings: citation based rankings (e.g., Azar and Brock 2008;Ritzberger 2008;Vieira 2008) and survey based rankings (e.g., Bräuninger and Haucap 2002;Hennig Thurau, Walsh, and Schrader 2004). Hybrid rankings, as a third ranking type, combine both approaches (e.g., Franke and Schreier 2008;Schulze, Warning, and Wiermann 2008). We will compare the different approaches in terms of strengths and weaknesses and explain how VHB JOURQUAL addresses them.
Citation based journal rankings such as the ISI Journal Citation Reports are often considered "ob jective" (e.g., Ritzberger 2008). Here, the ranking position of a journal depends on the number of citations the papers in the journal receive. The idea behind it is that citations are "the scientific commu nity's version of dollar voting by consumers for goods and services" (Laband and Piette 1994a: 641). Similar to the way economic theory considers pay ment at the cash desk as the result of a quality as sessment by consumers, a citation based ranking treats a citation as a proof of perceived quality. However, there are two major caveats to this ap proach, particularly when audiences are interested in the scientific quality of a journal (e.g., Schulze, Warning, and Wiermann 2008): • Quality vs. impact. Citations are not objective indicators for an article's scientific quality. The decision to accept a submitted article, making it available for citations, is subjective and not only determined by scientific quality (Blank 1991;Frey and Rost 2008;Laband and Piette 1994b;Starbuck 2004), and authors' decision to cite an article is, in addition to quality, also influenced by the article's type and topic (e.g., state of the art review), its shortcomings as well as citation cartel memberships (e.g., Fabel and Heße 1999). Thus, citation indexes do not measure the scientific quality but rather the impact of a journal. The ISI Journal Citation Reports ranks a journal according to its "impact factor", not to a quality index. Impact and scientific quality should not be treated as synonyms since the empirical correlations between them can be weak or even negative in some cases (e.g., Maier 2006;Schlinghoff and Backes Gellner 2002 . Considering all SSCI cate gories and other indexes mentioned above, these numbers will be only slightly higher (Clermont and Schmitz 2008;Dyckhoff and Schmitz 2007). In 2009, the ISI covered in total nine and Scopus 12 German language journals ranked in VHB JOURQUAL2, which have predominantly low ratings (Clermont 2009). Consequently, in German speaking countries citation based rankings are hardly able to measure publication productivity of business scholars (Albers 2009;Dilger 2000).
Survey based rankings also face limitations. How ever, these obstacles appear less systematic and strongly depend on the mode of data collection and analysis when compared to those of citation based rankings. Major threats to the validity of survey based rankings include the following issues: 1 • Sample. Respondents from whose assessments survey based rankings are constructed are not always suited for assessing the scientific quality of academic journals. VHB JOURQUAL aims to overcome this potential threat by including only VHB members who, at a minimum, hold a post doctoral position, with the majority being full professors. VHB JOURQUAL also requires active readership of a journal to rate its quality (it first asks the respondents to indicate which journals they have recently read and only in a second step asks respondents to evaluate those journals only) and uses an expertise factor to account for the respondents' differing levels of expertise (see also Heischmidt and Gordon 1993;Extejt and Smith 1990 danger of overstraining by a highly customized online survey design. Specifically, all journals are assigned to business sub disciplines (e.g., accounting, finance), and respondents are asked to evaluate only journal titles relevant for their specific sub discipline. • Strategic answers. Opponents of the survey based approach argue that researchers not always evaluate journals according to their actual quality perception, but rather in a way that is best for them (e.g., Schulze, Warning, and Wiermann 2008). Specifically, given that journal rankings can influence careers, scholars will have an interest that the journals they publish in or serve for as editors or reviewers are highly ranked; something which they can influence through their own ratings. As the incentive for such strategic answers strongly depends on the individual researcher's impact on a journal's rating, VHB JOURQUAL only considers journals with at least ten individual ratings. Moreover, ratings in VHB JOUR QUAL2 are not anonymous -each respondent had to use a unique ID and agree that his data could be linked to his personal information by the authors of this study. This should reduce the motivation for strategic answers, since evident over or underrating may become overt to the VHB JOURQUAL authors. In addition, outlier judgments were systematically removed as will be explained later in more detail.
Finally, hybrid rankings combine data from expert surveys and citation indices. This extends the num ber of journals and bases the evaluation on a broader foundation. However, the hybrid approach does not heal limitations inherent in the sources which serve as necessary inputs such as missing citation data for German language journals.

Measuring journal quality in VHB-JOURQUAL
Survey based rankings usually solely focus on the overall quality of the articles that are published by a certain journal. In VHB JOURQUAL, we conceptu alize overall scientific quality of a journal as being defined by two quality dimensions which are meas ured separately, namely the quality of the articles published in a journal (article quality) and the qual ity of the review process of the journal (review qual ity). Both quality dimensions are treated as forma tive indicators of overall journal quality (for a simi lar approach, see Rossiter 2002). Our distinction between article and review quality draws from qual ity literature in related fields such as service man agement, which has often been used as a template for higher education research (e.g., Hennig Thurau, Langer, and Hansen 2001). Service research distinguishes between outcome related quality aspects and those aspects which are related to the process of the service production as dimensions of overall (perceived) service quality (e.g., Brady and Cronin 2001). While article quality serves as the equivalent of outcome quality in the context of academic knowledge generation, review quality refers to the process of how articles (and their authors) are treated by the respective journal.
In that sense, article and review quality measure different dimensions of academic knowledge gen eration. Not all academics (as "customers") experi ence both quality dimensions; while article quality can be assessed by all readers of a journal's articles (including those who had only limited contact with a journal), the assessment of review quality requires deeper experiences and insight into a journal's in ternal processes. The inclusion of review quality not only helps to capture scientific quality in a more comprehensive way, but also adds dynamics and flexibility, as changes in the review process will be experienced much faster than their manifestation in printed articles. The VHB JOURQUAL index, on which the rating and ranking of journals is based, combines the two quality dimensions of article quality and review quality in a weighted additive composition (see equation 1). Both quality dimensions are measured with single items which is an adequate procedure for expert surveys with formative measures, as is the case with VHB JOURQUAL (Rossiter 2002). The scientific quality of the review process is defined as "standards for submissions requested by review ers and/or editors" (in German: "die Ansprüche, die Gutachter und/oder Schriftleiter bzw. Editoren an eingereichte Beiträge stellen"). Review quality is rated only by those who have either submitted at least one paper within the last five years or know the review process as reviewers or editors of the journal.
In VHB JOURQUAL1 only authors who had sub mitted papers were allowed to evaluate the review process. Also letting reviewers and editors evaluate review quality leads to a higher number of review process evaluations and reduces the likelihood that ratings and rankings are mainly or only based on article quality assessments. We weight the formative indicators of article quality and review quality equal (both aJ and bJ = 0.5), assuming both play the same role for determining overall scientific quality of a journal. However, to account for the limited number of review evalua tions for several journals and the loss in reliability associated with a small sample size for the review quality indicator, we adjusted the weights for article and review quality if the number of review quality assessments fell short of a certain threshold. Equa tion 2 lists the adjustments in weighting the two quality criteria for different numbers of ratings of a journal's review quality. If no respondent has rated a journal's review process, the weight for b becomes 0 and (since aJ = 1 -bJ) the overall JOURQUAL rating is solely based on the article quality indicator (with aJ = 1). (2) Expertise is included in the calculation of the VHB JOURQUAL2 overall quality score as a weighting factor since we expect a positive correlation between the expertise of a respondent and the validity of her or his assessment. We argue that scholars who have published in a variety of journals, and particularly in leading international journals, will be better able to judge the quality of articles and review processes than colleagues who lack that experience. Specifi cally, we operationalize expertise by three indica tors: (a) the number of different journals a respon dent has published in during the last five years; (b) the number of publications in high quality journals (defined as journals with an unweighted VHB JOURQUAL index value ≥ 7) during the last five years; and (c) the international experience gathered by publications in English language high quality journals during the last five years. For the first two indicators the raw values are transformed to scales ranging from 1 to 2; for indicator (c) respondents with at least one international high quality journal publication receive a 2, all others a 1. The three indi cators are multiplied and then rescaled to an exper tise scale ranging from 1 to 5. As a consequence, the evaluations of the respondents scoring with the highest expertise are weighted five times higher than the assessments of the respondents with the lowest expertise. We prefer a multiplicative function of expertise over an additive one, as we argue that academic expertise builds up in a non linear way, with the three indicators' impact on overall exper tise not being independent. While the journal ranking results directly from the different VHB JOURQUAL index values, the rating assigned to an index value requires a categorization. VHB JOURQUAL sorts journals into rating catego ries based on their index values, using absolute numbers as thresholds for defining the categories for the lack of a more objective categorization. Table  1 lists the six rating categories ranging for A+ to E and the respective VHB JOURQUAL scores.

Survey and sample of VHB-JOURQUAL2
The VHB JOURQUAL2 survey was conducted from February to April 2008. Every VHB member with a registered email address (N = 1,555) received an invitation email from the VHB chairmen and the authors of the study which contained a unique link to an online questionnaire. Every participant went through the highly individualized survey procedure depending on his or her specific research fields, reading habits, submission activities and reviewer or editor positions. 1,011 respondents started the evaluation process (response rate of 65 %), about 600 fully completed the questionnaire. Many re spondents who "dropped out" had actually pro ceeded through major parts of the survey. We in cluded all responses in our calculations regardless of technical completion. The questionnaire included a total of 1,633 journals. In addition to academic journals in a narrow sense, the list also comprises yearbooks and proceedings with a homogeneous review process. This is due to the observation that in some business sub disciplines (such as information systems) refereed conference proceedings fulfill similar tasks than actual journals. The list of journals for VHB JOURQUAL2 was selected in a multi step process. First, we synchronized the VHB JOURQUAL1 jour nal list with the established Harzing List of busi ness administration journals (Harzing 2007). Sec ond, we assigned the journals to different business sub disciplines which strongly overlapped with the 16 sections of the VHB ("Wissenschaftliche Kom missionen"; e.g., finance and banking, taxation, international management). Third, in a joint effort with all chairpersons of the VHB sections, we added new or previously overlooked journals, eliminated those which have ceased publication and adjusted the categorization. Forth, we invited all VHB mem bers to complete the journal list and asked them to indicate which journals they actually read in an online pre survey (n = 489). The final list then in cluded all journals which were either indicated by at least two respondents in the pre survey or evaluated by at least five respondents in VHB JOURQUAL1. Journals which did not meet one of these require ments were excluded as we expected them not to receive the minimum number of ten evaluations in the main survey required for the inclusion in the final ranking.
To reduce the number of strategic answers, we lim ited the anonymity of the study by informing the participants in the invitation mail that the authors of the study would be able to assign every rating to the individual respondent. Also, indicating the number of submissions to and publications in a journal was not sufficient; respondents also had to name respective papers' short titles. This informa tion was used to assure a valid calculation of expert factors and that review processes are only evaluated by respondents who have actually experienced them within the period of observation. While we are aware that these measures cannot completely re move all kinds of strategic behavior, they certainly increased the psychological barriers to behave in such a way.
To further improve the quality of our data, we ex cluded outlier ratings when calculating the VHB JOURQUAL2 ratings, as we assumed outliers to be based on misunderstandings or strategic misevalu ations. Specifically, for each journal, we kept all responses within the 99 % confidence interval (two sided) for both dimensions of quality (i.e. article and review quality) and deleted those responses outside the confidence interval. This procedure resulted in the removal of a total of 315 individual ratings. About 90 % of these deleted ratings were below the confidence interval; thus, the risk of overrating seems to be lower than the risk of underrating.

General Results
742 journals received more than 10 ratings in VHB JOURQUAL2; journals which received less than 10 ratings were excluded to assure a sufficient level of reliability (Web Appendix 1 contains the alphabeti cal list of all 742 journals). 2 As VHB JOURQUAL2 aims at business scholars, we only included journals in the ranking which -in addition to being read by at least 10 respondents -could be assigned to at least one sub discipline of business administration (e.g., accounting, marketing) or whose review proc ess was evaluated by a minimum of 5 respondents. 666 journals met at least one of these criteria and were subsequently included in the VHB JOURQUAL2 journal ranking. Figure 1 shows the distribution of VHB JOUR QUAL2 (JQ2) ratings for all 666 journals for the overall VHB JOURQUAL2 index value as well as the two quality dimensions and lists descriptive information. The mean of the overall JQ2 score is 6.22, equal to a C rating. Figure 2 displays the distributions for the quality weighting parameters and the respondent expertise parameter. The weighting factor for the two quality criteria varied between 0 and 0.5, with an average weighting for the review process of .24 (standard deviation = .18). As can be seen in Figure 3, the expertise factor for the respondents has a mean value of 1.36 (standard deviation = .58) and is positively skewed (the forth quartile ranges from 1.74 to 5), that is, relatively few researchers have very high expertise scores. This is consistent with the finding of Dyckhoff and Schmitz 2 Please note that this number differs from the initial results published on the VHB website as we deleted journals which were duplications of other journals with slightly different names or had ceased publication before the time frame considered in this ranking; we thank Robert Hofmeister from the Thurgau Institute of Economics at University of Konstanz for his valuable input. For duplicated journals, the evaluation scores were merged on an individual respondent level; if more than one journal version was rated by a respondent, the mean of his or her ratings for the respective journal was considered. As a result of this merging process, the scores for these journals can differ from those originally reported on the VHB website.

Figure 3: Distribution of Respondent Expertise Parameters in the Sample and Descriptive Statistics
The results also reveal that the double blind review ing process has become a condition sine qua non for a high quality journal among VHB members, fol lowing international standards. The journals with the highest decrease (by percentage) in quality ra ting did not employ a rigorous review process dur ing the last five years (Appendix 3 lists the journals and their respective changes since JQ1). Most of the journals with a high decrease in quality perception had already a below average rating in JQ1. Table 3 lists the best rated journals for 16 business administration sub disciplines; the disciplines were selected based on the VHB sections structure. The table also reveals to which extent the journal quality ratings are affected by judgments of researchers who do not belong to a specific sub discipline. In other words: Does the "core audience" of a journal judge its quality differently than other scholars?           Notes: AQ = article quality; RQ = review quality; n = number of evaluators; SM = section members. * No general management section exists within the VHB; ** Less than 10 journals with n ≥ 10 are considered as falling into this cate gory As can be seen, differences between quality ratings are usually very limited. Specifically, for the journals listed in Table 3, total ratings and those based on section members correlate with r = .91 (p < .01, n = 133) in the case of article quality and with r = .92 (p < .01, n = 96) for review quality. The average ratings are slightly higher for the section members (7.85 vs. 7.57 for article quality; 8.30 vs. 8.08 for review process quality), but the difference is not significant for any of the two quality indicators. In our model, we assume that article quality and review quality define the overall scientific quality of an academic journal. Both constitute different, but related dimensions of quality, as a high quality process will usually go hand in hand with high out come quality; something which is also reflected by a correlation of r = .75 between the two quality di mensions. The merit of measuring quality via the two dimensions becomes apparent when studying those journals for which both quality criteria differ substantially. Table 4 lists those journals whose articles and review standards are perceived most differently by the respondents. Among the journals whose articles receive better quality ratings than the review process are some which are explicitly positioned as "transfer" jour nals, linking scientific insights with managerial audiences (e.g., Harvard Business Manager, Sloan Management Review). Those journals which re ceive higher review ratings than article ratings, however, seem to have difficulties to fully transfer the quality of their review process into their final product, the published articles. An alternative ex planation might be that these journals have in creased the quality of their review through proce dural changes only recently, but the change has not reached the majority of the journals' readership yet, since article related ratings can be expected to be more resistant to change than review ratings.

Respondent-level Determinants of Quality Assessments
To learn which variables explain the interpersonal differences in journal quality ratings and to shed more light on the unobserved heterogeneity which underlies the aggregated results, we conducted an additional post hoc analysis. We focused on the three most prominent German language business research journals, namely Die Betriebswirtschaft -DBW, Zeitschrift für Betriebswirtschaft -ZfB, Schmalenbachs Zeitschrift für betriebswirtschaftli che Forschung -zfbf (Macharzina, Wolf, and Rohn 2004;Schlinghoff and Backes Gellner 2002). This selection offers two main advantages: these journals (a) are read by a large number of respondents, and they (b) contain articles from different subdisci plines of business administration, attracting a highly diverse readership. We conducted OLS regressions for each of the three journals, with the perceived article quality serving as dependent variable. As independent variables, we included the individual respondent's expertise fac tor, his or her affiliation with the 16 VHB sections which represent business administration sub disciplines (scholars can be affiliated with multiple sections), the respondent's status as a board mem ber of the respective journal, his or her academic status (i.e. full professor or not), as well as age and gender as demographic characteristics. The regres sion results are reported in Table 5.
The results show a substantial amount of overlap between the three journals; a correlation analysis with the standardized regression coefficients as cases exhibits correlations of r = .66 (DBW/zfbf), r = .68 (DBW/ZfB), and r = .84 (zfbf/ZfB). Consistent ly, corporate taxation researchers tend to rate the German business administration journals higher than scholars from other sub disciplines; the same is true for accounting scholars. These findings might be attributed to the fact that in both sub disciplines  the domestic legal framework plays a crucial role, so that research in these fields will have a stronger focus on national issues. Consequently, the relative importance of the domestic scientific community and of its German language journals might be higher than in other disciplines. As the top domestic journals are the best possible publication outlets for these researchers, competition for publication space and article quality might also be relatively higher. Furthermore, we see that scholars with higher levels of research expertise tend to rate the German language general business administration journals lower; the effect is significant for two of the three journals. Obviously, experience with review proces ses of international journals -a major facet of re search expertise -tends to heighten the researcher's comparison standard, resulting in lower evaluations for German business administration journals. Also, established scholars tend to hold a more critical attitude -ratings of full professors tend to be lower for the German business administration journals, with the effect being significant again for two of the three journals.

Validation of VHB-JOURQUAL2
To test the reliability and validity of VHB JOURQUAL2, we compare its results with a num ber of other international ratings: the first edition of  ) andfor the journals ranked in VHB JOURQUAL2 -also in Web Appendix 2 of this paper. The comparison with VHB JOURQUAL1 allows us to assess the reliability of the results, as the method and population for both rankings is very similar. Although differences will result from changes in journal quality over time and perception changes, they should be of a somewhat limited size. VHB JOURQUAL1 itself has been successfully validated in comparison with leading international survey based journal rankings (Hennig Thurau, Walsh, and Schrader 2004). For the 666 business administration journals in VHB JOURQUAL2, we collected the VHB JOURQUAL1 index values of journals in the final ranking (nJ ≥ 10). Data was available for 326 journals from VHB JOURQUAL1. We find that the correlation is significant and sub stantial, with r = .94 (p < .01).
Regarding the comparison between VHB JOUR QUAL2 and the ISI Impact Factors, it is important to see that both ratings measure related, but distinct constructs. While VHB JOURQUAL2 explicitly focuses on the scientific quality of a journal, ISI Impact Factors are an established measure which reflects the degree to which a journal's articles are read and actively cited by researchers. Since high quality articles are on average more likely to be cited by scholars than low quality ones (e.g., Hult, Rei mann, and Schilke 2009), a significant correlation between the two rankings can be expected. How ever, due to the conceptual differences between the two constructs, the correlation between VHB JOURQUAL2 and the ISI Impact Factors should be weaker than the correlation between the two ver sions of VHB JOURQUAL. We collected the impact factors from the ISI Journal Citation Report Edition 2008 (Thomson Reuters 2009) for the disciplines business, business/finance, and management. Data was available for 137 journals listed in VHB JOUR QUAL2. We find that the correlation between VHB JOURQUAL2 and the ISI Impact Factors is r = .57 (p < .01); it is slightly higher (r = .59) when quad ratic scores are used to account for the skewed dis tribution of the ISI Impact Factors. In addition to being significant and substantial, these correlations are also substantially lower than the one between VHB JOURQUAL2 and VHB JOURQUAL1, which is in line with our theoretical arguments. Finally, when comparing the VHB JOURQUAL2 ratings with the international journal rankings listed above, we ran pairwise comparisons and deter mined the correlation between VHB JOUQUAL2 and each ranking. When doing this, we included all journals which are considered in VHB JOUR QUAL2 and the respective comparison ranking (n = 329 for ABS09; n = 233 for CNR08; n = 207 for EJL06). In each case the correlation is strong and significant (p < .01) with r = .64 for ABS09, r = .70 for CNR08, and r = .56 for EJL06. It is again consis tent with our arguments that the correlation be tween VHB JOURQUAL2 and EJL06 is relatively weaker than between VHB JOURQUAL2 and the two other rankings, as EJL06 is partly based on citations.
In summary, we interpret these results as strong support for VHB JOURQUAL2's reliability and vali dity.

7
Discussion, Implications, and Future Research Perspectives

Discussion and Implications
This article reports the results of VHB JOUR QUAL2, a survey based ranking of 666 business administration journals, and details the underlying methodology. In addition to presenting the results for the different journals, we provide empirical evi dence for the ranking's reliability and validity. Thus, we have confidence that VHB JOURQUAL2 is a sound instrument to evaluate the journal publishing achievements of business researchers. As such, we believe that the major contribution of this ranking is its ability to reduce the level of arbitrariness and the importance of non performance related network characteristics (such as "academic provenance") from key decisions made at universities -some thing which will benefit both universities and good scholars. As its predecessor, VHB JOURQUAL2 carries the potential to stimulate business researchers in Ger many, Austria, and Switzerland to compete with colleagues from around the world for publication space in leading international journals, which are highlighted in this ranking. By doing this, VHB JOURQUAL2 might further raise the level of global competitiveness of the German speaking business administration community, a trend which we al ready see as a result of the existence of VHB JOURQUAL1 (Homburg 2008). Furthermore, we hope that the ranking will also help to make schol ars' intellectual achievements much easier to com municate to colleagues, department heads, deans, and rectors, a precondition for getting adequate rewards for such achievements. Nevertheless, we believe that the results reported here have to be treated with great care. We have ambiguous feelings when we read job postings for full professorships which say that the "scientific performance is mainly evaluated by number and quality of scientific publications in international journals according to the VHB JOURQUAL Rank ing" (like the University of Siegen in 2008, own translation). Although this indicates that VHB JOURQUAL indeed influences the community (and that we have reached an objective we had when we once initiated it), we see the danger that the impor tance of the VHB JOURQUAL ranking might be carried to excess. The scientific performance -not to say the overall performance -of an academic must not be solely judged on the basis of a single criterion, that is, a scholar's top journal publications according to VHB JOURQUAL. As Albers (2009: 361) states, "we should be aware that any ranking can only provide a small piece of the overall per formance picture". Business researchers have to be careful not to over emphasize the part of the picture which is measurable by ratings and rankings today, since the result would be counterproductive for the whole profession (Adler and Harzing 2009). As a consequence, we see a strong need for additional rankings of scholarly performances. Alternative ratings that measure business researchers' contribu tions in journal articles might use VHB JOURQUAL as a comprehensive and powerful source for inte grating survey based and citation based approaches into hybrid rankings; other rankings might want to emphasize a journal's reputation or its importance for knowledge transfer. The discrepancy between scientific quality and relevance for business manag ers has been shown by Oesterle (2006), who re ported a significant negative correlation between VHB JOURQUAL results and academic journal use by German managers with a PhD. In addition to different journal evaluations, powerful measure ment tools for books, teaching, or university man agement achievements would be valuable to avoid the threat of one dimensional university professors (Frey 2007). However, VHB JOURQUAL should not be blamed for its occasional misuse. It undisputedly covers a key facet of scholars' professional performance by providing a reliable and valid estimate of the scien tific quality of a business researcher's journal arti cles. It should be treated as such, no more, but also no less.

Future Research Perspectives
While the current state of VHB JOURQUAL pro vides an established tool for research performance evaluation, it also raises questions which should be considered as opportunities for future research.
Regarding the VHB JOURQUAL methodology, it is obvious that some elements are based on pragmatic considerations and might be considered arbitrary, at least to a certain extent. So we encourage future research to identify more theoretically and/or em pirically justifiable approaches for the following aspects of VHB JOURQUAL and compare their results with the current ranking: Definition and labeling of rating categories: In stead of measuring quality on a non labeled 10 point scale and assigning the categories ex post, the respondent might be asked to use the category la bels themselves. While this was not possible in the first edition of the ranking (and perhaps might have raised substantial problems even when collecting data for the second edition), the category labels are now widely established among German speaking business scholars. Alternatively, an empirical ap proach using empirical distributions might be cho sen to assign rating categories, which reduces the danger that journals fall directly below the threshold between two categories. Selection, weighting, and composition of indicators for the expertise factor: Regarding respondents' expertise, two relevant questions refer to the dimen sions of expertise and their composition: Should other factors than those currently represented by the expertise construct be considered when measur ing expertise? And is the current multiplicative composition optimal -and how would a different combination of expert dimensions affect the results? However, the current results show that the impact of the expertise factor should not be overrated, as expertise weighted results and unweighted results do not differ substantially for most journals.
Weighting of article and review quality: The cur rent version of VHB JOURQUAL posits that article and review quality are of equal importance for con structing the overall quality score of a journal (with adjustments if only a small number of judgments exists for review quality). An alternative approach would be to empirically determine the relevance (or factor weights) of the two quality dimensions. For example, conjoint measurement approaches can support (or reject) our decision for weighting of both determinants equal. Also, the weight correc tion for the review quality dimension in the case of limited review related judgments might be ques tioned and potentially improved. Especially the potentially strong effect of low numbers of review quality ratings which differs substantially from the usually much higher number of article ratings would deserve additional thought; maybe it would be ad vantageous to treat review assessments as outliers in such a case. At the same time, theoretical or em pirical arguments would be valuable to demonstrate the superiority of alternative approaches.

Minimum number of article quality evaluations:
While the current version of VHB JOURQUAL considers a minimum number of 10 ratings as the threshold for the inclusion of a journal, future re search might address whether this number is ade quate to guarantee sufficient reliability or if lower numbers are possible -or a higher number re quired.
In addition, future studies would be welcome which provide insight why most of the highest ranked journals come from marketing and finance. Both are large and global disciplines with a long tradition in journal ratings, which had also a large number of respondents in VHB JOURQUAL2. We tested for an impact of the number of raters of a journal on the journal's quality assessment, but found none within our sample (neither linear nor non linear). We sus pect that global competition for publication space is particularly strong in marketing and finance, which would contribute to the establishment of highly selective journals which are then perceived as of outstanding academic quality. The analysis of de terminants for three German language journals has shown that marketing and finance scholars do not have a general tendency to evaluate academic jour nals more positively than researchers from other business areas. It is inevitable that the need for justification and adequate foundation of a journal ranking increases with its importance. Thus, we see the success of VHB JOURQUAL as an obligation for its improve ment. However, the potential trade off between optimization and comparability of different JOUR QUAL editions needs careful considerations.