Advertisement

Network DEA: an application to analysis of academic performance

  • Mohammad Ali Saniee Monfared
  • Mahsa Safi
Open Access
Original research

Abstract

As governmental subsidies to universities are declining in recent years, sustaining excellence in academic performance and more efficient use of resources have become important issues for university stakeholders. To assess the academic performances and the utilization of the resources, two important issues need to be addressed, i.e., a capable methodology and a set of good performance indicators as we consider in this paper. In this paper, we propose a set of performance indicators to enable efficiency analysis of academic activities and apply a novel network DEA structure to account for subfunctional efficiencies such as teaching quality, research productivity, as well as the overall efficiency. We tested our approach on the efficiency analysis of academic colleges at Alzahra University in Iran.

Keywords

Data envelopment analysis Performance indicators Academic efficiency Network DEA 

Abbreviations

DEA

Data envelopment analysis

DMU

Decision-making units

NDEA

Network data envelopment analysis.

Introduction

Universities play a crucial role in the development of a country. The academic performances and more efficient use of resources have always been important issues for university top administrators, policy makers, state, faculties, employers, staffs, alumni, and students (Ahn et al. 1988, 1989; Abbott and Doucouliagos 2003; Avkiran 2001; Monfared et al. 2006). In order to assess the academic performances and the utilization of the resources, it appears that two important issues need to be addressed, i.e., a capable methodology and a set of good performance indicators as we consider in the following.

The evaluation of education with ranking lists of universities has become, over the past few decades, increasingly popular. These rankings have certainly serious impacts on universities' prestige and the number and quality of applicants, among other factors. The methods used in these league tables (Avkiran 2001; The Complete University Guide 2013) to rank universities have been criticized by many as they use a simple weighted sum of performances, which is not rigorous and has some methodological problems and limitations (Giannoulis and Ishizaka 2010).

Among all evaluation methods, it has been data envelopment analysis (DEA), a nonparametric linear programming technique, which may become a standard tool for evaluating efficiencies (Charnes et al. 1979; Cook and Seiford 2009). DEA is an approach for identifying best practices of peer decision-making units (DMUs) in the presence of multiple inputs and outputs. DEA not only provides efficiency scores for inefficient DMUs, but also provides for efficient projections for those units onto an efficient frontier. DEA computes a comparative ratio of weighted outputs to weighted inputs for each unit, which is reported as the relative efficiency score. The efficiency score is usually expressed as either a number between 0 and 1, or as a percentage. A DMU with a score of less than 1 is deemed inefficient relative to other units.

DEA has been applied across a wide range of industries as well as in nonprofit organizations. For a more recent and in-depth introductory treatment to DEA and for some mathematical representations of DEA, see the work of Charnes et al. (1979) and Cook et al. (2010). The DEA in particular has been widely used to evaluate performance and rank universities or schools (see, e.g., Abbott and Doucouliagos 2003; Ahn et al. 1988, 1989; Avkiran 2001; Celik and Ecer 2009; Cook and Seiford 2009; Gander 1995; Giannoulis and Ishizaka 2010; Heald and Geaughan 1994; Johnes 2006; Seiford 1996; Sinuany-Stern et al. 1994). As a matter of fact, education and higher education evaluation are in the fifth place of the most common field of study in DEA literature (Liu et al. 2013).

The focus for efficiency analysis in the education sector has been on teaching quality (see, e.g., Avkiran 2001;Celik and Ecer 2009), research productivity (see, e.g., Johnes and Yu, 2008; Kao and Hwang 2008; Tyagi et al. 2009), cost efficiency (see, e.g., Tyagi et al. 2009), or aggregate performance (see, e.g., Abbott and Doucouliagos 2003; Avkiran 2001; Colbert et al. 2000; Glass et al. 2006; Johnes 1992). However, all models developed so far are of the single-stage DEA type. Even when both teaching and research are considered together, they are aggregated to form a single-stage DEA model. These models neglect the internal linking activities, and thus, it is difficult to accurately evaluate the impact of teaching and research inefficiencies on the overall efficiency of the university.

In fact, single-stage DEA considers a DMU as a ‘black box’ and neglects intervening processes, i.e., different series or parallel functions. A novel DEA model called network DEA or NDEA that accounts for divisional efficiencies as well as the overall efficiency has been proposed in recent literature (see, e.g., Cook et al. 2010;Fare and Grosskopf 1996, 2000;Xie and Chong 2009). We think NDEA provides a capable methodology to assess different aspects of universities in a unified framework as we develop in this paper.

On the other hand, DEA and NDEA could work well if proper input and output indicators can be defined and used. Different indicators have been used in the assessment of universities throughout more than four decades of conducting investigations (see, e.g., Gander 1995; Cave et al. 1988; Johnes and Taylor 1990; Johnes 1992). Still, we witness immense variations in different works available in the current literature. As no definitive study could be found to guide the selection of performance indicators in the educational context of DEA, a closer examination of the literature encouraged us to advance a common denominator for performance indicators in academic efficiency analysis as will be reported, as a byproduct, in this paper.

This paper is organized as follows: in the ‘Specifying DMU's inputs and outputs’ section, the fundamental theory of measuring the efficiency of the universities, programs, colleges, and departments is reviewed. Important indicators to specify and measure the quality of teaching and research are discussed. The ‘Network DEA’ section develops a new network DEA structure to measure the relative efficiency of both teaching quality and research productivity. The ‘DEA mathematics’ section reviews the mathematics of DEA models, and the ‘Data and computation’ section provides test results and interpretations for performance efficiency analysis of Alzahra colleges. A conclusion is drawn in the last section.

Specifying DMU's inputs and outputs

Although teaching and research have been considered by most people as the two major tasks of the universities, they are difficult to measure. We need some input/output factors which are capable of representing the achievement of teaching quality and research productivity. We also need measures of the resources that the department has consumed in performing those two tasks. The selection of input and output factors for evaluating the performance of university departments using DEA has been discussed in several studies. A good review and discussion up to the year 1999 is the work of Avkiran in the analysis of Australian universities (Avkiran 2001). It is clear in this investigation that academic units' assessments by DEA models are very diverse, as both the types and the numbers of input–output factors are different. One wonders whether such situation is due to the fact that different universities have different traits and background or that an explanation must be sought in theory deficiency; that theory has not yet been adequately developed to assess universities effectively. In the following, we consider the latest developments in recent literature to present an updated view on input–output factors of importance in academic performance efficiency analysis.

Complications

It has been indicated by Higgins (1989) that the most difficult area in academic performance evaluation is that of teaching quality as the output performance of students may be the result of the students' initial ability which they have already acquired before entering the university. Student evaluation for teachers may also be biased by the nature of courses and does not have a common base for comparison if the students have not been taught by all teachers (Kao and Hung 2008). Shifting preferences may complicate measurement of a factor like student satisfaction. In 2004, a research conducted by Sabanci University (Celik and Ecer 2009) for the purpose of revealing the elements of the university preferences of the first 5,000 students who have the highest scores showed that 54% of the successful students did not feel happy with their universities and academic programs. In this framework, 23% of the students expressed their intention of changing their academic programs; 16% of students have a tendency for changing both university and academic program, and 10% of the students who were willing to change their universities complain about lack of good and qualified education. On the other hand, 14% of them complain about the lack of desirable facilities in their universities. The shifting preferences of most successful students in Turkey can also be found in Iran.

Complications also exist in the research side of the equation of academic performance analysis. As with the measurement of quality, there lacks a common base for comparing the quality of different research work, and subjectivity is usually involved. Typical performance indicators on the research side include number of publications, citation number, and journal impact factors. Number of publications, often interpreted as a measure of research productivity, suffers from the problem of different practices across disciplines. Citations attracted by an article has its drawbacks too, as articles in different disciplines have different shelf life, e.g., compare articles in mathematics which have longer shelf life than those in medicine. Impact factors are not problem-free either (Avkiran 2001).

In addition to teaching and research outputs, many believe that universities produce social output, an output in which there is no adequate measure for it (see, e.g., Higgins 1989; Johnes and Yu 2008; Kao and Hung 2008; Kao and Hwang 2008). Another difficulty is the availability of data. For example, some scholars suggest using the salary of the first job after graduation as a measure of the achievement of teaching. Unfortunately, these data are very difficult to acquire. Besides, different professions have different salary standards. It would be unfair to compare the salary of an elementary school teacher with that of a medical doctor (Kao and Hung 2008). Data dynamics is another factor of concern because any data refer only to a point in time, and some factors vary over time. For example, the ‘employability’ of a university's graduates depends on the economic situation.

Even in some studies (see, e.g., Colbert et al. 2000; Johnes 20062008; Kao and Hwang 2008; Tyagi 2009), various models have been proposed and compared, meaning that a single model cannot adequately analyze the efficiency. It is generally agreed that input–output factors should integrally reflect the purpose of the assessment; the factors in the same level should be relatively independent, i.e., contents of the factors should not be repeated in different forms (Meng et al. 2008). By reviewing the literature as summarized in Table 1, it is clear that much value is inherent in any agreement which proposes that inputs and outputs should be used to assess universities.
Table 1

Variations of DEA models in academia

Row

Author and year

Country

Number of proposed models

Number of inputs

Number of outputs

Level of study

Assessment perspective

1

Colbert et al. (2000)

USA

3

3

3-4

Program

Overall

2

Abbott and Doucouliagos (2003)

Australia

1

6

3

Universities

Overall

3

Avkiran (2001)

Australia

3

2

2-3

Universities

Teaching, overall

4

Glass et al. (2006)

UK

1

4

3

Universities

Overall

5

Johnes (2006)

UK

9

3-6

2-3

Universities

Overall

6

Kao and Hung (2008)

Taiwan

1

3

3

Departments

Overall

7

Johnes and Yu (2008)

China

4

6

3

Universities

Research

8

Meng et al. (2008)

China

3

3

3-8

Universities

Research

9

Xie and Chong (2009)

China

1

5

6

Colleges

Overall

10

Tyagi et al. (2009)

India

10

2-3

1-3

Departments, programs

Teaching, research, cost, overall

11

Celik and Ecer (2009)

Turkey

1

10

1

Program

Teaching

12

Katharaki and Katharakis (2010)

Greece

1

4

2

Universities

Overall

13

Kong and Fu (2012)

Taiwan

2

1

2

Colleges

Overall

The complications considered above is due to the fact that universities still retain their certain key characteristics that set them apart from other types of organizations. The key characteristics according to Lindsay (1982) are ‘the lack of profit motive, goal diversity and uncertainty, diffuse decision making, and poorly understood production technology’.

Proposed inputs and outputs

Our proposed set of performance indicators/input–output factors is illustrated in Table 2. These proposals are the first comprehensive attempt, to the best of our knowledge, to cover almost all factors of relevance suggested in the literature. It also comprises some new factors that we found important in our investigations, e.g., input factors 7 and 9 and output factors 5, 8, 9, and 15. However, some comments need to be made here. We do not mean that any efficiency analysis should cover all these factors as some may not be of relevance. For example, in normal programs of public-funded universities in Iran, the completion rate is almost unity, i.e., it is a very rare case if a student who successfully entered into a program, passing a very difficult entrance exam, could not complete the course. Therefore, completion rate cannot be a factor of importance and can safely be disregarded. Still, in the virtual programs which are also run by the public-funded universities in Iran, the completion rate could be of relevance and need to be considered.
Table 2

Proposed inputs and outputs for DEA model

Outputs

Inputs

1. Teaching load

1. Number of BSc students (O)

2. Students assessments: teaching quality, retention rate (I)

2. Number of MSc and PhD students (O)

3. Graduate prospects: employability of a university's graduate, e.g., average starting salary, recruiter satisfaction with team players, employment rate

3. Number of fee-paying students or income from tuitions (O)

4. Good honors: percentage of undergrads achieving an entry into graduate programs

4. Student's satisfaction from program and university, e.g., percentage of alumni who donate, satisfaction (O) with curriculum, satisfaction with placement

5. Quality score of admitting universities

5. Entry standards: students' initial ability or student minimum entrance exam, average GMAT score

6. Completion: completion rate of those studying at the university

6. Number of faculty members (instructors, assistant professors)

7. Publications (journal papers, conference papers, books)

7. Number of faculty members (associate professors, professors)

8. Hot papers (in top research journals)

8. Faculty member's positions (weighted average)

9. Invited keynote speech in international conferences

9. Number of adjunct professors

10. Grants and external contracts (education, applied research, consultancy)

10. Number of administrative staff

11. Donations (I)

11. Floor space

12. Awards for education, research, and community services(at both the national and the international levels)

12. Values of laboratory assets

13. Inventive patent

13. Budget per student

14. Scientists' cultivation (excellent leaders, internationally known scientists)

14. Total incomes (O)

15. Social and economic contributions (community services)

15. Research budget per faculty member (O)

Also, note that in Table 2, we have considered that some of our proposed inputs can also be considered as outputs, denoted by (O), depending on the perspective under which the efficiency analysis is performed. For example, ‘Research budget per faculty member (O),’ i.e., our proposed input number 15, which is an input factor in an overall efficiency analysis can become an output factor in a research productivity analysis. On the other hand, some of our outputs factors listed in Table 2 can become input factors denoted by (I). For example, ‘Donations (I),’ i.e., our proposed output number 11, which is an output for a highly reputed university run by the private sector, can become an input factor for a publicly funded university.

However, Table 2 contains 15 input factors as well as 15 output factors which seem too many as compared with the current researches exemplified in Table 1. The reason why researchers prefer DEA models with a very limited number of input and output factors is an important subject for further research. An important reason is the so-called golden rule. According to the golden rule, the number of DMUs divided by three could be a good indicator of the numbers of input and output in any DEA models. For example, if we have nine DMUs, as is the case in our analysis of assessing Alzahra colleges, we could have three input and outputs all together. However, the golden rule is compromised in actual modeling efforts to provide a better tradeoff between the golden rule and an effective DEA model. We therefore suggest that the input–output factors developed in Table 2 are considered as a good starting point in selecting the most relevant input–output factors.

Network DEA

The single-stage DEA models are based on thinking about production technology as a black box that transforms inputs into outputs. However, in most real situations, the DMUs may perform several different functions and can also be separated into different components either in series or in parallel and/or in a more complex form of network type. In such situations, some components play important roles in producing outputs through the use of intermediate outputs obtained from their previous components. Here, the black box approach provides no insights regarding the inter-relationships among the components' inefficiencies and cannot provide specific process guidance to DMU managers to help them improve the DMU's efficiency. Thus, adding some degrees of structure might suit the application better. However, network DEA accounts for divisional efficiencies as well as the overall efficiency in a unified framework (see, e.g., Cook et al. 2010; Fare and Grosskopf 19962000; Lindsay 1982; Xie and Chong 2009).

Teaching and research in academia are examples where up to the present time, simple aggregated single-stage models have been used. The aggregation of teaching and research neglects the internal linking activities, and thus, we could not evaluate the impact of teaching and research inefficiencies on the overall efficiency of the university. Furthermore, the single-stage model might choose an inappropriate pair of input vs. output for evaluation and assign an unreasonable score to the concerned DMU, since DEA selects the most favorable pair for the DMU in the sense of maximizing the ratio scale. In other words, the analysis does not fully access the underlying diagnostic value potentially available to the management. The single-stage models also rouse a problem involving the degree of freedom in that the number of input and output items increases relative to the number of DMUs.

This is where teaching and research activities and their associated input–output factors differ by nature, though they are not mutually exclusive. In fact, they are two independent objects with obvious overlaps. For example, a faculty member who teaches a course at an undergraduate level is more a teacher rather than a researcher. Also, a faculty member who teaches a supporting course for graduate students is both a teacher and a researcher.

In our study of Alzahra's academic colleges and by considering the availability of data, we have proposed a two-stage NDEA model as shown in Figure 1, where the first stage defines the teaching activities and the second stage defines the research activities. Here, x 1 tr Open image in new window is the number of lecturers and assistant professors; x 1 tr Open image in new window is the number of associate professors and professors; y 1 t Open image in new window is the number of undergraduate students; y 1 tr Open image in new window is the number of graduate students, i.e., master students and PhD students; y 1 r Open image in new window denotes the total internal and external grants; and finally, y 2 r Open image in new window is the number of awarded researchers and the number of presented papers at international venues. The awarded researchers are those faculty members who have been selected by the office of the vice president for research as active faculties with intensive research endeavors. Those members of faculties who attended international conferences and presented papers have also been selected as active researchers, and we have taken their records as a measure of research activity. The sum of these records for any college is taken as a measure of research productivity.
Figure 1

Teaching and research efficiency model: a network DEA.

In our proposed NDEA model, the faculty members, i.e., x 1 tr Open image in new window and x 2 tr Open image in new window are considered as input factors both to the teaching box and to the research box, with the ratio of 1/3 and 2/3 as shown in Figure 1. This means that one third of a faculty member timing is assigned to teaching courses and two thirds of it is assigned to research tasks. In the research box, y 1 tr Open image in new window is an intermediate measure, as it is an output factor for the teaching box and an input factor to the research box. The intermediate measure y 1 tr Open image in new window is the number of students who have completed their undergraduate studies and have been admitted into the graduate program. These students are input resources to our research box as they undertake research activities at the graduate level. In the standard DEA model, we get an efficiency score of unity for all DMUs if there is a measure that is to be treated as both an input and output, i.e., an intermediate measure.

To present a more complete model, we could add a new input variable denoted as x 1 r Open image in new window which is the number of students who are admitted into the graduate program and who have completed their undergraduate studies elsewhere. These students along with those who have finished their undergraduate studies in Alzahra University are in fact input factors to our research box. However, as the number of DMUs or colleges we have in Alzahra University is only nine, we have to keep the number of inputs and outputs proportional. Hence, in the present paper, we did not consider x 1 r Open image in new window as an independent input factor to our research box. As a rule of thumb, the number of DMUs should be more than threefold the sum of inputs and outputs. Here, if we want to consider this rule, the number of DMUs should be increased or the number of input and output should be decreased. The first idea is not applicable as there are only nine colleges in Alzahra University. On the other hand, to follow the second idea, we had to have only three indicators which are too little, causing poor analysis and less discriminating power. The decreased discrimination produces too many efficient DMUs which is also unrealistic. For this reason, we have to make a compromise as we did in our network DEA model as depicted in Figure 1.

In order to provide the necessary background for testing our network DEA model, we briefly review DEA mathematics in the following section.

DEA mathematics

The mathematical formulations for different DEA models are not discussed because others have already adequately covered this (Cook and Seiford 2009; Cooper et al. 2006). Here, we only give a brief introduction of basic DEA mathematics in order to present the mathematics needed for network DEA models.

Consider that we have n DMUs where each has m inputs denoted as y10, y20,…, ym 0 and s outputs denoted as y10, y20,…, ym 0. Weights assigned to outputs are u1, u2,…, u s , and weights assigned to inputs are v1, v2,…, v m . The efficiency is defined as a fraction.
r = 1 s u r y r 0 i = 1 m v i x i 0 Open image in new window
(1)
Then, the basic model of DEA is called the CCR ratio model which is defined as (2):
Max Z 0 = r = 1 s u r y r 0 i = 1 m v i x i 0 St r = 1 s u r y rj i = 1 m v i x ij 1 , for j = 1 , , n ; r = 1 , , s ; i = 1 , , m ; u r , v i 0 Open image in new window
(2)
This is referred to as constant returns to scale model. This fractional nonlinear program model of (2) can be converted into a linear program of (3) which is called the multiplicative form of the output-oriented CCR model:
Min Z 0 = r = 1 s v i x i 0 st r = 1 s u r y r 0 = 1 ; i = 1 m v i x ij r = 1 s u r y rj 0 , j = 1 , , n ; r = 1 , , s ; i = 1 , , m ; u r , v i 0. Open image in new window
(3)
Its dual which is called envelopment form of the output-oriented CCR model is presented as (4):
Max y 0 = θ st j = 1 n λ j x j x i 0 , for i = 1 , 2 , , m ; j = 1 n λ j y rj θ y r 0 , for r = 1 , 2 , , s ; λ j 0 , for j = 1 , 2 , , n ; θ unrestricted in sign . Open image in new window
(4)
Here, θ is the dual variable for constraint r = 1 s u r y r 0 Open image in new window, and the dual variable for constraint i = 1 m v i x ij r = 1 s u r y rj 0 Open image in new window is λ j . In (5), we present the equivalent output-oriented envelopment form for the BCC model that is referred to as variable return to scale model:
Max y 0 = θ st j = 1 n λ j x ij x i 0 , for i = 1 , 2 , , m ; j = 1 n λ j y rj θ y r 0 , for r = 1 , 2 , , s ; j = 1 n λ j = 1 , for j = 1 , 2 , , n ; λ j 0 , for j = 1 , 2 , , n ; θ unrestricted in sign. Open image in new window
(5)
Now, let us consider a two-stage process where x ij are inputs to the first stage, z dj are outputs from the first stage and at the same time inputs to the second stage (z dj is the d th intermediate variables of the J th DMU), and y rj are outputs from the second stage. Using the network approach of Fare and Grosskopf (1996), we can write the network model (6) as follows (Cook et al. 2010):
Min Z 0 = θ st j = 1 n λ j x ij θ x ij 0 , for i = 1 , 2 , , m ; j = 1 n λ j μ j z dj 0 , for d = 1 , 2 , , D ; j = 1 n μ j y rj y rj 0 , for r = 1 , 2 , , s ; λ j , μ j 0 , for j = 1 , 2 , , n ; θ unrestricted in sign. Open image in new window
(6)
Model (6) is the dual (Cook et al. 2010) to the so-called centralized model (7) as:
Max y 0 = r = 1 s u r y r 0 st i = 1 m v i x i 0 = 1 ; r = 1 s u r y rj d = 1 D w d z dj 0 , for j = 0 , 1 , 2 , , n ; d = 1 D w d z dj i = 1 m v i x ij 0 , for j = 0 , 1 , 2 , , n ; w d 0 , for d = 1 , 2 , , D ; v i 0 , for i = 1 , 2 , , m ; u r 0 , for r = 1 , 2 , s. Open image in new window
(7)

This model is the Kao and Hwang model (2008).

Data and computation

Alzahra University has 9 colleges and 41 departments of which 34 departments offer baccalaureate degrees, 34 offer masters degrees, and 10 offer doctoral degrees. The Department of Mathematics, the Department of Foreign Languages, and the Department of Islamic Teachings offer courses such as calculus, English, and ethics to other departments, respectively. These departments are called service departments, and courses are called service courses. It should be noted that the Women Research Center has no teaching duties, i.e., it produces only research products associated with the social life of women. There are some departments that do not offer teaching at the undergraduate level including the Department of Linguistics, the Department of Information Technology in Engineering, the Department of Educational Psychology, the Department of Information Technology in Management, and the Department of Art Research. Data for input–output factors for nine colleges of Alzahra University are illustrated in Table 3.
Table 3

Inputs-outputs statistics for colleges of Alzahra

Row

College name

Inputs for teaching and research

Output for teaching

Output for teaching/input for research

Outputs for research

  

x 1 tr Open image in new window

x 2 tr Open image in new window

y 1 t Open image in new window

y 1 tr Open image in new window

y 1 r Open image in new window

y 2 r Open image in new window

1

Literature, Languages and History

31

12

681

235

2.3 + 0

16 + 15

2

Theology and Islamic Teachings

27

1

507

179

4 + 0

15 + 0

3

Women Research Center

4

0

0

0

2 + 2

6 + 6

4

Sports

12

0

480

38

2.3 + 0

4 + 6

5

Social and Economic Sciences

36

9

1,044

342

4 + 3.7

16 + 15

6

Basic Sciences

55

27

1,295

433

23.7 + 2.33

37 + 37

7

Psychology and Education

29

9

706

310

8.7 + 0.33

16 + 13

8

Engineering

12

1

405

84

1.7 + 4.3

5 + 1

9

Arts

29

1

959

228

8 + 1.7

16 + 6

The following notes should be made with respect to Table 3:

(1). Data for inputs and outputs identified in Table 3 were extracted from publications by the vice president for research for years 2000 to 2006 (Vice president for research 20022005);

(2). Data in the seventh column, e.g., 2.3 + 0 in row 1, represent the internal grants and the external grants, respectively. Also, data in the eighth column, e.g., 16 + 15 in row 1 represents the number of awarded researchers and the number of conference papers presented at international venues, respectively. It should be noted that the act of adding these two different numbers together provides us with a better picture of the research productivity of colleges in Alzahra. It also helps to achieve better efficiency scores as we realize that the number of DMUs in this study is only nine and that there should be some appropriate correspondence between the sample size and sum of the number of inputs and outputs;

(3). Data for internal grants and external grants for the Woman Research Center is estimated by the authors.

The results for our network DEA model, as shown in Figure 1, are illustrated in Table 4. We have also solved three alternative single-stage models as shown in Figure 2 including the single-stage teaching model, single-stage research model, and single-stage aggregated model. Test results for these alternative models are summarized in columns 2, 3, and 4 of Table 4. We have solved the linear program of relevance 36 times, i.e., 9 times for each model for 4 different models (see Appendix for an instance of the network DEA linear program). All four different performance models in this study run under variable return to scale.
Figure 2

Single-stage models.

Table 4

Efficiency scores in four different models

Row

Single-stage teaching model

Single-stage research model

Single-stage aggregated model

Network model

1

0.79

0.69

0.87

0.73

2

0.86

0.66

0.86

0.83

3

1

0.66

0.25

0.22

4

1

0.84

1

1

5

1

0.74

1

0.92

6

0.82

0.98

1

0.76

7

1

0.82

1

1

8

1

1

1

0.81

9

1

0.84

1

1

It is interesting to note that with our traditional single-stage aggregated model, we have six colleges that perform efficiently, while using our advanced network DEA model, we have only three efficient colleges. In DEA approach, this implies that network DEA discriminatory power is better. Assuming that the network model is subsumed to the single-stage aggregated model, we search for correspondence between efficient colleges. Therefore, a college that is efficient in the network model is expected to be efficient in the single-stage aggregated model.

Examination of Table 4 also reveals that three out of six efficient colleges in the single-stage aggregated model are also efficient in the network model. Similarly, efficient colleges in the single-stage teaching model are efficient in the network model. However, similar results do not hold for the single-stage research model. Such observations indicate a considerable level of discriminatory power. Comparisons of scores between the network model and single-stage aggregated model are shown in Figure 3. This result emphasizes that the traditional DEA model which treats the efficiency evaluation like a black box and ignores the internal processes is not appropriate for identifying inefficient DMUs and evaluating the degree of their inefficiencies since it provides little insight into the inefficient sources and the locations where the inefficiency may occur.
Figure 3

Comparisons of scores between two models.

Comparing the results obtained for the single-stage teaching model and the results obtained for the single-stage research model, using the existing data, one can conclude that Alzahra's overall teaching quality is better than its research productivity. Still, we believe that data for more indicators as suggested in Table 3 should be collected so that a more precise picture can appear, illustrating the academic performance of Alzahra University. An analysis at the department level, in particular, can shed more light on the working situation and ups and downs of Alzahra University.

Conclusions

Higher education plays a key role in the development of a country. With the increasing number of young people enrolling in higher education programs in different universities, the quality of university degree has become increasingly a matter of concern for academic professors, university administrators, policy makers, and the government. The main objective of this study was to develop a novel two-stage network DEA model to examine the relative efficiency of teaching quality and research productivity of higher education institutes. We have also developed an appropriate set of input–output factors for performance analysis of universities.

We tested our network DEA model on the assessment of colleges of Alzahra University against three alternative single-stage models. The results reveal that the discriminatory power of our network DEA is stronger than those of the single-stage models, i.e., the efficiency scores can better indicate which colleges need further attention, but it does not say in what area. On the other hand, our functional analysis using single-stage models will provide efficiency scores for teaching quality and research productivity. These results together give a superior picture of the existing inefficiencies and their relevant areas unmatched with the traditional single-stage models. As this is the first paper which attempts to develop a new network structure for productivity analysis of higher education institutes, we think further investigations should be undertaken in both theoretical and experimental aspects to strengthen our findings.

Appendix

Full description of NDEA model for the College of Literature, Languages and History (see formula number (6) in the ‘DEA mathematics’ section and Table 3 in the ‘Data and computation’ section):

Min θ

Subject to

10.33λ1 + 9λ2 + 1.33λ3 + 4λ4 + 12λ5 + 18.33λ6 + 9.66λ7 + 3λ8 + 9.66λ9 − 10.33θ ≥ 0

4λ1 + 0.333λ2 + 0λ3 + 0λ4 + 3λ5 + 9λ6 + 3λ7 + 0.333λ8 + 0.333λ9 − 4θ ≤ 0

681λ1 + 507λ2 + 0λ3 + 480λ4 + 1,044λ5 + 1,295λ6 + 706λ7 + 405λ8 + 959λ9 ≥ 681

235λ1 + 179λ2 + 0λ3 + 38λ4 + 342λ5 + 433λ6 + 310λ7 + 84λ8 + 228λ9 ≥ 235

20.66μ1 + 18μ2 + 2.66μ3 + 8μ4 + 24μ5 + 36.66μ6 + 19.33μ7 + 8μ8 + 19.33μ9 ≤ 20.66

8μ1 + 0.666μ2 + 0μ3 + 0μ4 + 6μ5 + 18μ6 + 6μ7 + 0.666μ8 + 0.666μ9 ≤ 8

681μ1 + 507μ2 + 0μ3 + 480μ4 + 1,044μ5 + 1,295μ6 + 706μ7 + 405μ8 + 959μ9 ≤ 681

235μ1 + 179μ2 + 0μ3 + 38μ4 + 342μ5 + 433μ6 + 310μ7 + 84μ8 + 228μ9 ≤ 235

2.3μ1 + 4μ2 + 4μ3 + 2.3μ4 + 7.7μ5 + 26.03μ6 + 9.03μ7 + 6μ8 + 9.7μ9 ≥ 2.3

31μ1 + 15μ2 + 12μ3 + 10μ4 + 31μ5 + 74μ6 + 29μ7 + 6μ8 + 22μ9 ≥ 31

λ1 ≥ 0 λ2 ≥ 0 λ3 ≥ 0 λ4 ≥ 0 λ5 ≥ 0 λ6 ≥ 0 λ7 ≥ 0 λ8 ≥ 0 λ9 ≥ 0

μ1≥ 0 μ2 ≥ 0 μ3 ≥ 0 μ4 ≥ 0 μ5 ≥ 0 μ6 ≥ 0 μ7 ≥ 0 μ8 ≥ 0 μ 9≥ 0

The solutions are as follows:

θ = 0.73, λ1 = λ2 = λ3 = λ4 = λ5 = λ6 = 0, λ7 = 0.5732, λ8 = 0.68, λ9 = 0, μ1 = 1, μ2 = μ3 = μ4 = μ5 = μ6 = μ7 = μ8 = μ9 = 0.

Notes

Acknowledgment

The authors are grateful for many insightful comments that they have received from two anonymous referees, who helped bring the presentation of this paper to the present state.

Supplementary material

40092_2012_42_MOESM1_ESM.tif (430 kb)
Authors’ original file for figure 1
40092_2012_42_MOESM2_ESM.tif (1.1 mb)
Authors’ original file for figure 2
40092_2012_42_MOESM3_ESM.tif (303 kb)
Authors’ original file for figure 3

References

  1. 1.
    Abbott M, Doucouliagos C: The efficiency of Australian universities: a data envelopment analysis. Econ Educ Rev 2003, 22: 89–97. 10.1016/S0272-7757(01)00068-1CrossRefGoogle Scholar
  2. 2.
    Ahn T, Charnes A, Cooper WW: Some statistical and DEA evaluations of relative efficiencies of public and private institutions of higher learning. Socioecon Plann Sci 1988, 22: 259–269. 10.1016/0038-0121(88)90008-0CrossRefGoogle Scholar
  3. 3.
    Ahn T, ArnoldV CA, Cooper WW: DEA and ratio efficiency analyses for public institutions of higher learning in Texas. Res Govern Nonprofit Account 1989, 5: 165–185.Google Scholar
  4. 4.
    Avkiran NK: Investigating technical and scale efficiencies of Australian universities through data envelopment analysis. Socioecon Plann Sci 2001, 35: 57–80. 10.1016/S0038-0121(00)00010-0CrossRefGoogle Scholar
  5. 5.
    Cave M, Hanney S, Kogan M, Trevett G: The use of performance indicators in higher education: a critical analysis of developing practice. London: Jessica Kingsley; 1988.Google Scholar
  6. 6.
    Celik O, Ecer A: Efficiency in accounting education: evidence from Turkish universities. Crit Perspect Account 2009, 20: 614–634. 10.1016/j.cpa.2008.01.007CrossRefGoogle Scholar
  7. 7.
    Charnes A, Cooper WW, Rhodes E: Short communication: measuring efficiency of decision making units. Eur J Oper Res 1979, 3: 339. 10.1016/0377-2217(79)90229-7CrossRefGoogle Scholar
  8. 8.
    Colbert A, Levary RR, Shaner MC: Determining the relative efficiency of MBA programs using DEA. Eur J Oper Res 2000, 125: 656–669. 10.1016/S0377-2217(99)00275-1zbMATHCrossRefGoogle Scholar
  9. 9.
    Cook WD, Seiford LM: Data envelopment analysis (DEA) - thirty years on. Eur J Oper Res 2009, 192: 1–17. 10.1016/j.ejor.2008.01.032zbMATHMathSciNetCrossRefGoogle Scholar
  10. 10.
    Cook WD, Liang L, Zhu J: Measuring performance of two-stage network structures by DEA: a review and future perspective. Omega 2010,38(6):423–430. 10.1016/j.omega.2009.12.001CrossRefGoogle Scholar
  11. 11.
    Cooper WW, Seiford LM, Tone K: Introduction to data envelopment analysis and its uses with DEA-solver software and references. New York: Springer; 2006.Google Scholar
  12. 12.
    Fare R, Grosskopf S: Productivity and intermediate products: a frontier approach. Econ Lett 1996, 50: 65–70. 10.1016/0165-1765(95)00729-6CrossRefGoogle Scholar
  13. 13.
    Fare R, Grosskopf S: Network DEA. Socioecon Plann Sci 2000, 34: 35–49. 10.1016/S0038-0121(99)00012-9CrossRefGoogle Scholar
  14. 14.
    Gander JP: Academic research and teaching productivities: a case study. Technol Forecast Soc Chang 1995, 49: 311–319. 10.1016/0040-1625(95)00054-ECrossRefGoogle Scholar
  15. 15.
    Giannoulis C, Ishizaka A: A web-based decision support system with ELECTRE III for a personalized ranking of British universities. Decis Support Syst 2010, 48: 488–497. 10.1016/j.dss.2009.06.008CrossRefGoogle Scholar
  16. 16.
    Glass JC, McCallion G, McKillop DG, Rasaratnam S, Stringer KS: Implications of variant efficiency measures for policy evaluations in UK higher education. Socioecon Plann Sci 2006, 40: 119–142. 10.1016/j.seps.2004.10.004CrossRefGoogle Scholar
  17. 17.
    Heald D, Geaughan N: Formula funding of UK higher education: rationales, design and probable consequences. Financ Account Manage 1994, 13: 267–289.CrossRefGoogle Scholar
  18. 18.
    Higgins JC: Performance measurement in universities. Eur J Oper Res 1989, 38: 358–368. 10.1016/0377-2217(89)90012-XCrossRefGoogle Scholar
  19. 19.
    Johnes G: Performance indicators in higher education: a survey of recent work. Oxford Rev Econ Policy 1992, 8: 19–34. 10.1093/oxrep/8.2.19CrossRefGoogle Scholar
  20. 20.
    Johnes J: Data envelopment analysis and its application to the measurement of efficiency in higher education. Econ Educ Rev 2006,25(3):273–288. 10.1016/j.econedurev.2005.02.005CrossRefGoogle Scholar
  21. 21.
    Johnes G, Taylor T: Performance indicators in higher education. Buckingham: SRHE & Open University Press; 1990.Google Scholar
  22. 22.
    Johnes J, Yu L: Measuring the research performance of Chinese higher education institutions using data envelopment analysis. China Econ Rev 2008,19(4):679–696. 10.1016/j.chieco.2008.08.004CrossRefGoogle Scholar
  23. 23.
    Kao C, Hwang SN: Efficiency decomposition in two-stage data envelopment analysis: an application to non-life insurance companies in Taiwan. Eur J Oper Res 2008,185(1):418–429. 10.1016/j.ejor.2006.11.041zbMATHCrossRefGoogle Scholar
  24. 24.
    Kao C, Hung HT: Efficiency analysis of university departments: an empirical study. Omega 2008, 36: 653–664. 10.1016/j.omega.2006.02.003CrossRefGoogle Scholar
  25. 25.
    Katharaki M, Katharakis G: A comparative assessment of Greek universities' efficiency using quantitative analysis. Int J Educ Res 2010, 49: 115–128. 10.1016/j.ijer.2010.11.001CrossRefGoogle Scholar
  26. 26.
    Kong W, Fu T: Assessing the performance of business colleges in Taiwan using data envelopment analysis and student based value-added performance indicators. Omega 2012, 40: 541–549. 10.1016/j.omega.2011.10.004CrossRefGoogle Scholar
  27. 27.
    Lindsay AW: Institutional performance in higher education: the efficiency dimension. Rev Educ Res 1982,52(2):175–199. 10.3102/00346543052002175MathSciNetCrossRefGoogle Scholar
  28. 28.
    LiuJ LL, Lu W, Lin B: A survey of DEA applications. Omega 2013, 41: 893–902. 10.1016/j.omega.2012.11.004CrossRefGoogle Scholar
  29. 29.
    Meng W, Zhang D, Qi L, Liu W: Two-level DEA approaches in research evaluation. Omega 2008, 36: 950–957. 10.1016/j.omega.2007.12.005CrossRefGoogle Scholar
  30. 30.
    Monfared MAS, Gharneh NS, Mirkhani SN: Ranking analysis and modeling of state run universities. Scientia Iranica 2006,13(1):91–104.Google Scholar
  31. 31.
    Seiford LM: Data envelopment analysis: the evolution of the state of the art (1978–1995). J Product Anal 1996, 7: 99–137. 10.1007/BF00157037CrossRefGoogle Scholar
  32. 32.
    Sinuany-Stern Z, Mehrez A, Barboy A: Academic departments efficiency via DEA. Comput Oper Res 1994, 21: 543–556. 10.1016/0305-0548(94)90103-1zbMATHCrossRefGoogle Scholar
  33. 33.
    The Complete University Guide: The league table of UK universities. 2013. . Accessed 15 May 2013 http://www.thecompleteuniversityguide.co.uk . Accessed 15 May 2013Google Scholar
  34. 34.
    Tyagi P, Yadav SP, Singh SP: Relative performance of academic departments using DEA with sensitivity analysis. Eval Plann 2009, 32: 168–177. 10.1016/j.evalprogplan.2008.10.002CrossRefGoogle Scholar
  35. 35.
    Vice president for research: Research performance (2000–2001), Publication no. 98. Tehran (in Farsi): Alzahra University; 2002.Google Scholar
  36. 36.
    Vice president for research: Research indicators in year 2006, Publication no. 154. Tehran (in Farsi): Alzahra University; 2005.Google Scholar
  37. 37.
    Xie MW, Chong G: Evaluation of university creative talent cultivation based on modified data envelopment analysis model. Intnal Forum Info Tech Appl, IEEE Comput Soc 2009. 10.11091IFITA.292Google Scholar

Copyright information

© Saniee Monfared and Safi; licensee Springer. 2013

This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Authors and Affiliations

  1. 1.School of EngineeringAlzahra UniversityTehranIran
  2. 2.Department of Industrial EngineeringK.N. Toosi University of TechnologyTehranIran

Personalised recommendations