Earth Science Informatics

, Volume 10, Issue 3, pp 369–382 | Cite as

A multi-criteria method for assessment of spatial data infrastructure effectiveness

Open Access
Methodology Article

Abstract

There is a demand for evaluation of spatial data infrastructures (SDIs), to justify and monitor the relations between investments in SDI initiatives and the results obtained. It is also essential to pay particular attention to identifying user communities, and eliciting their assessment of effects deriving from an SDI. The paper introduces a concept of a multi-criteria method which allows to assess effectiveness of the SDI from the user perspective. The application of the proposed method in the Polish Spatial Data Infrastructure (PSDI) and its main access point to spatial data and services called ‘Geoportal 2’ as well as two groups of the national geoportal”s users (spatial planners and land surveyors) presents its potential. The total scores for the Geoportal 2 indicated the investment has potential and is quite effective, although some components of the PSDI (e.g. main access point, datasets, network services, software, hardware, procedures) may need improvements and additional analyses in the future. The contribution of this paper is the multi-criteria method which enables the analysis of outcomes, benefits (impacts) and business value of using SDI business project’s artifacts (outputs) considering the following dimensions: information and support provided, use process, user organizational performance, strategic alignment and business impact on user enterprise.

Keywords

User perspective Spatial data infrastructure (SDI) usefulness Spatial data infrastructure (SDI) usability Business project IT investment Multi-view SDI assessment framework 

Introduction

The increasing popularity of spatial data infrastructures (SDIs) has resulted in building new infrastructures on different levels of government (i.e. local, national, supranational), but also in developing the existing ones through such means as including new data sources. A growing body of literature describes many different aspects of SDI initiatives, including the concepts and models of SDIs (e.g. Hjelmager et al. 2008; de By et al. 2009), perspectives (e.g. Chan et al. 2001), development (e.g. Rajabifard et al. 2002), managing metadata, spatial data sets and services (e.g. Innerebner et al. 2016), as well as standards and implementing rules (e.g. Wortman 1994).

One of the important issues within the development of spatial data infrastructures (SDIs) is economic and financial analysis. There is a demand for SDI evaluation in order to (e.g. Grus et al. 2008) justify and monitor in a systematic way the relationships between the investments in SDI initiatives and the results obtained. Many different assessment methods are considered in the area of the SDI projects. To structure and organize SDI assessment the framework which integrates various approaches and techniques is proposed by Grus et al. 2008. Secondly, financial methods are used for SDI investments(e.g. Craglia and Nowak 2006; Bregt 2012). Moreover, there are examples of application of different qualitative methods, which come from different scientific fields, including operational research, management and economy (e.g. Geudens et al. 2009; Toomanian et al. 2011).

Questions about the efficiency and effectiveness of various SDI activities are also valid in monitoring not only the development, but also transitions and changes of SDI. Lance et al. (2006) note that efficiency, rather than effectiveness, is more commonly stated as an objective of SDI initiatives. However, in the literature, deliberations on effectiveness concerning both concept and scope in the field of SDI as well as indicators and assessment methods can be found. In the area of effectiveness assessment there are references to management and organization studies (Kok and van Loenen 2005; Giff and Crompvoets 2008; Grus et al. 2011), economic studies (Craglia and Campagna 2010), as well as to information system theory (Steudler et al. 2008; Nedović-Budić et al. 2008).

Indicators for measuring effectiveness are a separate issue. Steudler et al. (2008) derive evaluation and performance indicators for SDIs from land administration principles. Evaluation involves assessing the strengths and weaknesses of programs, policies, personnel, products and organizations to improve their effectiveness. The work of Giff and Crompvoets (2008) describes the framework of designing the performance indicators for assessing the efficiency and effectiveness of SDI, which are based on organizational development theory. Craglia and Campagna (2010) present example indicators for SDI effectiveness, butalso internal efficiency benefits of SDI, and the effectiveness benefits derived by the public and local businesses in dealing with their local public administration.

Nedović-Budić et al. (2008) present the evaluation framework focusing on the effective use of SDI, by identifying the current and potential users and finding out how useful the SDI-supplied data and services are or would be for their particular needs. It derives measures mainly from information systems that are based on concepts such as: usefulness, effective use, information and organizational effectiveness. The latter is also considered by Kok and van Loenen (2005).

There is also a conceptual logic model of an SDI component, which illustrates the relationships between an SDI’s inputs, outputs, outcomes, and impact within the context of efficiency and effectiveness as discussed in Giff and Crompvoets (2008).

The aim of the paper is to present a multi-criteria method for assessment of SDI effectiveness from users’ perspectives as the review of existing literature on this subject(e.g. Nedović-Budić et al. 2008;Vandenbrucke et al. 2013; Craglia and Nowak 2006) confirms the need for identifying user communities and assessment of value deriving froman SDI use. Undoubtedly the successful application and use of SDI products and services (Nedović-Budić et al. 2008) rely on the fulfillment of the users’ needs and expectations. In the course of this paper,a case study of two user groups – spatial planners and surveyors of the Polish Spatial Data Infrastructure (PSDI) – is taken into consideration. The assessment enables analysis of effects related to the PSDI business project carried out by the Polish Head Office of Geodesy and Cartography between 2008 and 2015 and concerns the project’s artifacts such as a geoportal (main access point to the PSDI) and its applications, datasets, as well as support provided.

The contribution of this paper is the qualitative method for ex-post assessment of SDI effectiveness based on the analysis of outcomes, benefits (impacts) and business value of using SDI business project’s artifacts (outputs), which includes accessibility, usefulness and usability of the project’s artifacts (e.g. main access point and its applications, spatial datasets, network services) as well as impact of the SDI on user organizational performance and business strategy.

The article is structured as follows. The Methodology section gives a brief outline of effectiveness assessment in this study, as well as presenting the concept of a method, which allows for the analysis of the dimensions of SDI effectiveness. The application of the method to spatial data and services of the PSDI, in Geoportal 2, the main Polish access point, is described in the Case study section. The results are then discussed and the solutions evaluated. A final summary follows in the conclusion.

Methodology

SDI business project

SDI could be considered as (PMBOK 2000) a temporary endeavor which is undertaken to create unique products and services. The SDI project is an IT-oriented project the purpose of which is design, implementation and installation of artifacts such as: computers, storage, networking and other physical devices, as well as software, databases and processes, and thereby constitutes the components of the SDI. Also essential for IT projects are (Murphy 2002) the business context and objectives of an organization and investors who are planning a project, have a project in progress or have just completed one. The organizations and investors implementing an SDI are mainly government authorities. However, the business context and objectives are as important to authorities as to private entities. Therefore, an SDI business project should be considered as the sum of information technology components to be designed, implemented, and installed, a whole IT lifecycle and the business objectives.

A viewpoint of the business project concerns SDI assessment in a frame of an (Zwirowicz-Rutkowska 2014) SDI business project approach, which uses the assumptions of the Multi-View SDI Assessment Framework (Fig. 1). The essence of this framework is (Grus et al. 2007, 2008) that it accepts multiple perspectives (views) on SDI and thus accepts its complexity in terms of multiple definitions. Moreover, each approach covers at least one of the three purposes of the assessment: accountability, knowledge and development. All approaches use one or more assessment methods, such as case studies, surveys, document analysis, etc., to evaluate SDIs. The application component of the assessment framework also focuseson measuring the indicators of each assessment approach. The result part of the framework has two functions: 1) evaluating the SDI and 2) evaluating each approach and the whole assessment framework.
Fig. 1

The scope of research (dark gray) against the multi-view SDI assessment framework

Scope of effectiveness assessment

The proposed method assessment is considered for the SDI business project approach which includes measuring SDI effectiveness from the perspective of the users and their organizational performance, as well as the organization undertaking the SDI investment.

Many authors indicate (e.g. Heffron 1989; Bannister 2001; Productivity Commission 2013) the multi-dimensionality and difficulty of measuring the concept of effectiveness. The term is not always defined or interpreted consistently within and across disciplines. It also concerns the area of SDI. Theeffectiveness issue is discussed in the field of SDI as it was presented in Introduction section, but the key aspects which influence effectiveness interpretation are features which should be evaluated, methods and indicators selection, as well as some terminological assumptions concerning the concept of spatial data infrastructure and the infrastructure specificity.

This study investigates effectiveness from the perspective of the users. In this case effectiveness is manifested by outcomes, benefits (impacts) and business value of using the SDI projects’ artifacts (outputs, e.g. main access point to the infrastructure and other applications, datasets, network services, metadata, hardware), which are described on four levels: 1) the SDI provides some data sources, applications and metadata which can be utilized by users and also supports the users in utilizing the functionality of the SDI, 2) the use of SDI components influence the decision making processes of the users and 3) the use of SDI components influence user organizational performance, as well as 4) the SDI might help to achieve users’ strategic goals and have an impact on their ability to transform business processes. Effectiveness refers to the extent to which performance of users’ duties and also their needs are supported and covered by the SDI components in terms of usefulness, accessibility and usability of the SDI projects’ artifacts. It also concerns achievement of business objectives of organizational units utilizing the SDI. The proposed scope of effectiveness derives from concepts of information effectiveness theory, including (Nedović-Budić et al. 2008) effective use, organizational effectiveness, user satisfaction, value of geographic information, operational effectiveness, as well as (Hamilton and Chervany 1981) system effectiveness, and also (Murphy 2002) business value from information technology.

Multi-criteria method characteristics

In the area of evaluation of IT initiatives, many different methods are presented, which could be categorized (Lech 2005; Renkema and Berghout 1997) as follows:1) financial methods, 2) qualitative methods (multi-criteria methods, strategic analysis methods, 3) probabilistic methods. For the presented business project approach, a multi-criteria method is proposed. Generally, (Lech 2005) multi-criteria methods used for IT projects like Information Economics (Parker and Benson 1988) or the method presented by T. Murphy (2002) try to evaluate all aspects of a single IT project. They are thus IT-oriented, have a wide evaluation scope (beyond financial aspect) and a single-project level of observation. They integrate some qualitative contributions with a wide range of quantitative non-financial aspects. Multi-criteria methods characteristics are as follows: 1) pillars, categories or domains of assessment with weighting schemas are identified, 2) indicators are grouped by each pillar, category or domain, 3) ranking, weighting or scoring schemas for indicators are assumed, 4) the results of the evaluation process are scores for each indicator, then pillar and also the total score for the project, which is then being interpreted.

Assessment categories

A proposed multi-criteria method dedicated to an SDI is for ex-post and monitoring assessment of effectiveness through four categories (Table 1), which adhere to the four levels mentioned presented in the Scope of effectiveness assessment subsection. Each category (C) is weighted (Wc) in accordance with their relative importance to the organization which undertakes the assessment. The sum of the weighed categories (Wp) is 100%.
Table 1

Assessment categories

Category ID

C

W(%)

1

Information and support provided

Wc1

2

Use process

Wc2

3

User organizational performance

Wc3

4

Strategic alignment and business impact on user enterprise

Wc4

SDI Project \( Wp=\sum_{i=1}^4{WC}_i \)

100

Indicators for assessment categories

Each category is split into n indicators (I), against which each project should be assessed (Tables 2, 3, 4, and 5). For each criterion, a 0–10 scale is proposed. For all categories, a higher score means that the project fulfills more of each criterion. As the SDI and information systems have much in common (e.g. hardware, applications), there are measures (column I in Tables 3 and 4), which derive from the field of operational research, benefits of IT projects and management information system (Hamilton and Chervany 1981; Farbey et al. 1992). There are also indicators (column I in Table 5), which adhere to aspects of business value from information technology (Murphy 2002). Some indicators (column I in Tables 2 and 3) are related to the general concept of the SDI outputs and to several aspects of effectiveness described by Nedović-Budić et al. (2004, 2008) and Zwirowicz-Rutkowska (2016).
Table 2

Scores for the category of information and support provided

C

I

ASI1 (spatial planners)

ASI2 (land surveyors)

WASI

σWASI

1.Information and support provided

1_1 Data -Thematic accuracy

7.0

7.4

7.2

±0.2

1_2 Data - Completeness

7.0

6.9

7.0

±0.0

1_3 Data - Spatial resolution

4.0

5.7

4.7

±0.8

1_4 Data - Temporal validity

4.6

4.3

4.5

±0.1

1_5 Data - Positional accuracy

5.0

4.9

5.0

±0.0

1_6 Data - Distribution format

3.0

5.7

4.2

±1.0

1_7 Data - Lineage

9.0

4.9

7.2

±2.0

1_8 Support - Help menu - FAQ

2.5

5.5

3.8

±1.5

1_9 Support - Help menu - Video tutorials

3.0

4.0

3.4

±0.5

1_10 Support - Contact menu - Email address

5.6

2.0

4.1

±1.8

1_11 Support - Contact menu - Telephone number

3.0

2.0

2.6

±0.5

1_12 Support - Forum menu

5.9

1.0

3.8

±2.4

1_13 Support - User’s manual

2.1

3.0

2.5

±0.4

1_14 Support - Other materials

2.0

3.0

2.4

±0.5

k = 2; WU1 = 28; WU2 = 21

ASc1 = 4.4

 

\( {\sigma}_{ASc_1} \)

±0.4

 
Table 3

Scores for the category of use process

C

I

ASI1 (spatial planners)

ASI2 (land surveyors)

WASI

σWASI

2. Use process

Decision makers

2_1 Higher confidence in making decisions

6.2

4.4

5.4

±0.9

2_2 More independent of suppliers, superior, other employees

7.8

3.8

6.1

±2.0

2_3 Easier/On-line user authorization

7.6

5.9

6.9

±0.8

2_4 Better understanding of problems and factors

8.0

6.0

7.1

±1.0

2_5 Increased comfort at work

7.8

5.7

6.9

±1.0

2_6 Improved competencies

6.8

2.6

5.0

±1.7

Decision making process

2_7 Detecting gaps in problem analysis

6.8

4.4

5.8

±1.2

2_8 Better information quality

7.5

5.0

6.4

±1.2

2_9 Faster access to information

8.1

4.6

6.6

±1.7

2_10 Access to more sources of information

8.3

2.8

5.9

±2.7

2_11 Easier task/goal formulation and realization

7.1

2.5

5.1

±2.3

2_12 Consideration of constraints and alternatives

6.0

2.1

4.3

±1.9

2_13 Length of time to make decision

8.0

6.3

7.3

±0.8

2_14 Length of time to acquire data

7.8

4.6

6.4

±1.6

2_15 Length of time to analyze data

8.0

5.6

7.0

±1.2

2_16 Thoroughly studies and analysis

6.8

4.9

6.0

±0.9

2_17 Better data management

6.7

3.5

5.3

±1.6

2_18 Better quality of decisions

6.8

5.4

6.2

±0.7

2_19 Better/Easier cooperation with different stakeholders

6.6

2.1

4.7

±2.2

2_20 Better/easier cooperation within an organization

6.3

3.0

4.9

±1.6

Applications

2_21 Accessibility

6.5

5.8

6.2

±0.3

2_22 Usability (intuitiveness, clarity and content presentation)

5.9

6.2

6.0

±0.1

2_23 Usefulness

6.3

6.8

6.5

±0.2

k = 2; WU1 = 28; WU2 = 21

ASc2 = 6.0

 

\( {\sigma}_{ASc_2} \)

±0.2

 
Table 4

Scores for the category of User organizational performance

C

I

ASI1 (spatial planners)

ASI2 (land surveyors)

WASI

σWASI

3. User organizational performance

3_1 Duration of procedure

7.3

4.4

6.1

±1.3

3_2 Change of attitude towards some procedures/tasks

6.6

4.5

5.7

±1.0

3_3 Improved procedures

7.5

4.8

6.3

±1.3

3_4 More executed plans, decisions, studies

6.2

1.6

4.2

±2.3

3_5 Increase in orders

5.8

1.2

3.8

±2.3

3_6 Automates manual calculation, analysis, tasks realization

6.3

1.9

4.4

±1.9

3_7 Automates data acquisition and collection

5.8

1.8

4.1

±2.0

3_8 The prompt completion of work

4.6

1.3

3.2

±1.6

3_9 Reduces costs of data acquiring and processing

5.8

1.7

4.0

±2.0

3_10 Cost displacement (e.g. software, hardware, people)

3.1

1.0

2.2

±1.0

3_11 Increase in costs of equipment, the infrastructure

3.1

1.0

2.2

±1.0

k = 2; WU1 = 28; WU2 = 21

ASc3 = 4.2

 

\( {\sigma}_{ASc_3} \)

±0.4

 
Table 5

Scores for the Strategic alignment and business impact on user enterprise category

C

I

ASI1 (spatial planners)

ASI2 (land surveyors)

WASI

σWASI

4. Strategic alignment and business impact

4_1 Corporate or brand image/public perception

2.6

1.3

2.0

±0.6

4_2 Improved understanding of competitive landscape

3.5

1.7

2.7

±0.9

4_3 Increase of the competitiveness of the firm

2.2

1.3

1.8

±0.4

4_4 Expand the firm’s offer with new services

3.4

1.5

2.6

±0.9

4_5 Formalizes innovation

4.2

2.4

3.4

±0.9

4_6 Improved knowledge transfer

7.0

4.5

5.9

±1.0

4_7 Enhance ICT and GIS/CAD knowledge

5.0

3.2

4.2

±0.9

4_8 Development of the firm

5.4

2.6

4.2

±1.4

4_9 Enhances linkages with customers and data suppliers

4.0

1.3

2.8

±1.3

4_10 Supports new communication and distribution channels (e-service, e-administration, e-business)

4.0

2.0

3.1

±1.0

4_11 Ability of ICT to cope with changing business processes

5.6

1.3

3.8

±2.1

4_12 Optimization of workflow

7.0

3.5

5.5

±1.7

4_13 Flexibility to reflect new business requirements

5.0

1.4

3.5

±1.8

4_14 Improved coordination in an organization

3.6

2.4

3.1

±0.6

4_15 Improved coordination with different participants of the tasks and procedures

5.0

2.4

3.9

±1.3

4_16 Increase of tasks/procedures/work supported by ICT

4.8

1.4

3.3

±1.7

4_17 ICT impact on the office/company’s organizational structures (new positions)

2.2

1.5

1.9

±0.3

4_18 Possibility of ICT inclusion in tasks

8.4

3.5

6.3

±2.4

4_19 ICT impact on efficiency increase of the employees and the whole company

7.6

3.5

5.8

±2.0

k = 2; WU1 = 28; WU2 = 21

ASc4 = 3.7

 

\( {\sigma}_{ASc_4} \)

±0.3

 

After scoring (SI) each indicator, the average score (ASc) and weighted score (WSc) for each category is calculated.

If there are m scores for each indicator (the group interview), the average score (ASI) for each indicator is calculated:\( ASI=\frac{1}{m}\sum_{i=1}^m{SI}_i \).

If there are k groups of users, a weight for each group is assigned (wu), which is equal to the number of respondents and then the weighted average score (WASI) for each indicator is calculated:\( WASI=\sum_{i=1}^k\frac{wu_i\times {ASI}_i}{wu_i} \).

For the purpose of accuracy evaluation of the averages standard deviation (σ) calculations are done as presented below.

A general equation for the standard deviation of the average score (σAS):
$$ {\sigma}_{AS}=\pm \sqrt{\frac{\sum_{i=1}^l{\nu_i}^2}{l\times \left( l-1\right)}} $$

where ν is a deviation from the average score (AS), lis a number of observations.

A deviation from the average score for each category:
$$ {\nu}_i= ASc-{WASI}_i $$
Evaluation of the average score calculations for each category:
$$ \sum_{i=1}^l{\nu}_i=0 $$

where l is a number of indicators.

A general equation for the standard deviation of the weighted average score for each indicator (σWASI):
$$ {\sigma}_{WASI}=\pm \sqrt{\frac{\sum_{i=1}^k{wu}_i\times {\nu_i}^2}{\sum_{i=1}^k{wu}_i\times \left( k-1\right)}} $$

where wu is a weight for each group, ν is a deviation from the weighted average score (WASI), k is a number of the group observations.

A deviation from the weighted average score for each indicator:
$$ {\nu}_i= WASI-{ASI}_i $$
Evaluation of the weighted average score calculations for each indicator:
$$ \sum_{i=1}^k{wu}_i\times {\nu}_i=0 $$

Total business project score and its interpretation

The total SDI business project weighted score (WSBP) is as follows:\( {WS}_{BP}=\sum_{i=1}^4{ WS c}_i \).

There are no absolute rules for interpreting the overall score, but for the method dedicated to SDI presented in this paper, based on the assumptions for IT projects (Murphy 2002), the propositions of interpretation of the total SDI business project weighted score are as follows. SDI projects scoring about 25% are rather poor and their artifacts are ineffective. Projects with scores between 26 and 50% have some potential but their artifacts require significant modification in the following projects. Projects with scores between 51 and 75% are good and quite effective, although their artifacts may need some improvements and additional analyses in the future. A project above 76% is a solid investment and its artifacts are of a high level of effectiveness.

Case study

The polish spatial data Infrastructure and the geoportal 2 business project

The Polish SDI is implemented by twelve different central government bodies. The PSDI is the part of Infrastructure for Spatial Information in the in the European Community (INSPIRE) and compatible with implementing rules, requirements and obligations established at Community level. According to the INSPIRE Directive (European Parliament and the Council 2007), the PSDI covers thirty-four INSPIRE spatial data themes. The PSDI is also developed at regional and local levels through different initiatives, and by many institutions and local authorities. The national geoportal (national access point) integrates certain thematic, regional and local geoportals and is the main access point to the PSDI resources.

The national access point, called ‘Geoportal 2,’ offers the following applications: a geoportal for the INSPIRE (INSPIRE broker), a national geoportal (national broker), a metadata editor and validator, mobile applications, and a statistics module and allows the access to a wide range of datasets, network services and metadata.

Applications and other mentioned SDI components are artifacts of the Geoportal 2 project, which was carried out by the Polish Head Office of Geodesy and Cartography between 2008 and 2012, finally closed in 2015. The IT life cycle of this business project included the following phases (Polish Head Office of Geodesy and Cartography 2009): 1) documentation preparation for funding, 2) project management, 3) development and maintenance of the PSDI, 4) development and maintenance of the Land Surveying and Cartographic Infrastructure, 5) implementation and maintenance of INSPIRE network services and INSPIRE broker, 6) implementation and maintenance of trade network services and national broker, 7) training, 8) promotion. The business context of this SDI project included (Polish Head Office of Geodesy and Cartography 2010) analysis of the Land Surveying and Cartographic Service’s tasks and data resources and other stakeholders’ tasks and resources, which build the Land Surveying and Cartographic Databases and Infrastructure, as a reference part of the National Spatial Data Infrastructure, as well as interactions between the Land Surveying and Cartographic Service and other government bodies engaged in the process of the PSDI implementation in three dimensions: European, national and trade. Business objectives included 1) enabling access and use of geographical information in Poland through the development of the national spatial data infrastructure in the part of the reference databases as well as network services associated with these datasets, 2) use of the electronic geodetic and cartographic archive and also other documents in the process of registers, maps and databases creation and maintenance for the purpose of development of the NSDI, 3) better access to geodetic and cartographic documents and their prevention of damage in case of the random incidents as well as deterioration of their condition as the result of the running use, 4) drawing up and implementation of the security politics which define the rules of the common spatial data and metadata use through the main access point as well as restriction procedures of the risk of the intentional or accidental contravene of spatial data and metadata confidentiality, integrity and accessibility, 5) implementation of the best practice in the system’s maintenance.

Application of the method

In this study, two groups of users of the PSDI and its main access point (Geoportal 2) are considered as follows: spatial planners and land surveyors.

The application of the method is based on surveys conducted between July 2014 and May 2015. The survey encompassed ten topics and different types of closed questions concerning the issues of users’ duties performance and their needs in relation with assessment of usefulness, accessibility and usability of the main access point, and referring to categories described in the Assessment Categories section (Table 1) and including the indicators presented in the Scores for categories section (Tables 2, 3, 4, and 5). A total of 63 questionnaires were mailed to companies specializing in urban and local level planning. An additional 100 questionnaires were send or presented to land surveying companies. The completion rate was 44.4% and 21%, respectively.

Results

Scores for categories

This section presents calculations of the average scores (ASI) and the weighted average score (WASI) for each indicator in the category, including 2 groups of user (k = 2) as well as the average scores for the assessment categories (Asc), and also standard deviations (σ) for averages (Tables 2, 3, 4, and 5). The base for calculations were results of surveys– scores for indicators indicated by 28 spatial planners and 21 land surveyors. Weights for groups are respectively WU1 = 28, WU2 = 21.Tables 6 and 7 show evaluation of the averages calculations.
Table 6

Evaluation of the weighted average scores calculations for each indicator

I

WASI

V1

V2

wu1 × v1

wu2 × v2

\( \sum_{i=1}^k{wu}_i\times {\nu}_i \)

I

WASI

V1

V2

wu1 × v1

wu2 × v2

\( \sum_{i=1}^k{wu}_i\times {\nu}_i \)

1.Information and support provided

2. Use process

1

7.17

0.17

-0.23

4.76

-4.83

-0.07 ≈ 0

1

5.43

-0.77

1.03

-21.56

21.63

0.07 ≈ 0

2

6.96

-0.04

0.06

-1.12

1.26

0.14 ≈ 0

2

6.09

-1.71

2.29

-47.88

48.09

0.21 ≈ 0

3

4.73

0.73

-0.97

20.44

-20.37

0.07 ≈ 0

3

6.87

-0.73

0.97

-20.44

20.37

-0.07 ≈ 0

4

4.47

-0.13

0.17

-3.64

3.57

-0.07 ≈ 0

4

7.14

-0.86

1.14

-24.08

23.94

-0.14 ≈ 0

5

4.96

-0.04

0.06

-1.12

1.26

0.14 ≈ 0

5

6.90

-0.90

1.20

-25.20

25.20

0.00 ≈ 0

6

4.16

1.16

-1.54

32.48

-32.34

0.14 ≈ 0

6

5.00

-1.80

2.40

-50.40

50.40

0.00 ≈ 0

7

7.24

-1.76

2.34

-49.28

49.14

-0.14 ≈ 0

7

5.77

-1.03

1.37

-28.84

28.77

-0.07 ≈ 0

8

3.79

1.29

-1.71

36.12

-35.91

0.21 ≈ 0

8

6.43

-1.07

1.43

-29.96

30.03

0.07 ≈ 0

9

3.43

0.43

-0.57

12.04

-11.97

0.07 ≈ 0

9

6.60

-1.50

2.00

-42.00

42.00

0.00 ≈ 0

10

4.06

-1.54

2.06

-43.12

43.26

0.14 ≈ 0

10

5.94

-2.36

3.14

-66.08

65.94

-0.14 ≈ 0

11

2.57

-0.43

0.57

-12.04

11.97

-0.07 ≈ 0

11

5.13

-1.97

2.63

-55.16

55.23

0.07 ≈ 0

12

3.80

-2.10

2.80

-58.80

58.80

0.00 ≈ 0

12

4.33

-1.67

2.23

-46.76

46.83

0.07 ≈ 0

13

2.49

0.39

-0.51

10.92

-10.71

0.21 ≈ 0

13

7.27

-0.73

0.97

-20.44

20.37

-0.07 ≈ 0

14

2.43

0.43

-0.57

12.04

-11.97

0.07 ≈ 0

14

6.43

-1.37

1.83

-38.36

38.43

0.07 ≈ 0

Asc1 = 4.447

Asc2 = 6.005

15

6.97

-1.03

1.37

-28.84

28.77

-0.07 ≈ 0

16

5.99

-0.81

1.09

-22.68

22.89

0.21 ≈ 0

17

5.33

-1.37

1.83

-38.36

38.43

0.07 ≈ 0

18

6.20

-0.60

0.80

-16.80

16.80

0.00 ≈ 0

19

4.67

-1.93

2.57

-54.04

53.97

-0.07 ≈ 0

20

4.89

-1.41

1.89

-39.48

39.69

0.21 ≈ 0

21

6.20

-0.30

0.40

-8.40

8.40

0.00 ≈ 0

22

6.03

0.13

-0.17

3.64

-3.57

0.07 ≈ 0

23

6.51

0.21

-0.29

5.88

-6.09

-0.21 ≈ 0

I

WASI

V1

V2

wu1 × v1

wu2 × v2

\( \sum_{i=1}^k{wu}_i\times {\nu}_i \)

I

WASI

V1

V2

wu1 × v1

wu2 × v2

\( \sum_{i=1}^k{wu}_i\times {\nu}_i \)

3. User organizational performance

4. Strategic alignment and business impact on user enterprise

1

6.06

-1.24

1.66

-34.72

34.86

0.14 ≈ 0

1

2.04

-0.56

0.74

-15.68

15.54

-0.14 ≈ 0

2

5.70

-0.90

1.20

-25.20

25.20

0.00 ≈ 0

2

2.73

-0.77

1.03

-21.56

21.63

0.07 ≈ 0

3

6.34

-1.16

1.54

-32.48

32.34

-0.14 ≈ 0

3

1.81

-0.39

0.51

-10.92

10.71

-0.21 ≈ 0

4

4.23

-1.97

2.63

-55.16

55.23

0.07 ≈ 0

4

2.59

-0.81

1.09

-22.68

22.89

0.21 ≈ 0

5

3.83

-1.97

2.63

-55.16

55.23

0.07 ≈ 0

5

3.43

-0.77

1.03

-21.56

21.63

0.07 ≈ 0

6

4.41

-1.89

2.51

-52.92

52.71

-0.21 ≈ 0

6

5.93

-1.07

1.43

-29.96

30.03

0.07 ≈ 0

7

4.09

-1.71

2.29

-47.88

48.09

0.21 ≈ 0

7

4.23

-0.77

1.03

-21.56

21.63

0.07 ≈ 0

8

3.19

-1.41

1.89

-39.48

39.69

0.21 ≈ 0

8

4.20

-1.20

1.60

-33.60

33.60

0.00 ≈ 0

9

4.04

-1.76

2.34

-49.28

49.14

-0.14 ≈ 0

9

2.84

-1.16

1.54

-32.48

32.34

-0.14 ≈ 0

10

2.20

-0.90

1.20

-25.20

25.20

0.00 ≈ 0

10

3.14

-0.86

1.14

-24.08

23.94

-0.14 ≈ 0

11

2.20

-0.90

1.20

-25.20

25.20

0.00 ≈ 0

11

3.76

-1.84

2.46

-51.52

51.66

0.14 ≈ 0

Asc3 = 4.208

Asc4 = 3.685

12

5.50

-1.50

2.00

-42.00

42.00

0.00 ≈ 0

13

3.46

-1.54

2.06

-43.12

43.26

0.14 ≈ 0

14

3.09

-0.51

0.69

-14.28

14.49

0.21 ≈ 0

15

3.89

-1.11

1.49

-31.08

31.29

0.21 ≈ 0

16

3.34

-1.46

1.94

-40.88

40.74

-0.14 ≈ 0

17

1.90

-0.30

0.40

-8.40

8.40

0.00 ≈ 0

18

6.30

-2.10

2.80

-58.80

58.80

0.00 ≈ 0

19

5.84

-1.76

2.34

-49.28

49.14

-0.14 ≈ 0

Table 7

Evaluation of the average scores calculations for each category

C

I

v

C

I

v

1.Information and support provided

1

-2.723

2. Use process

1

0.575

2

-2.513

2

-0.085

3

-0.283

3

-0.865

4

-0.023

4

-1.135

5

-0.513

5

-0.895

6

0.287

6

1.005

7

-2.793

7

0.235

8

0.657

8

-0.425

9

1.017

9

-0.595

10

0.387

10

0.065

11

1.877

11

0.875

12

0.647

12

1.675

13

1.957

13

-1.265

14

2.017

14

-0.425

15

-0.965

16

0.015

17

0.675

18

-0.195

19

1.335

20

1.115

21

-0.195

22

-0.025

23

-0.505

  

\( \sum_{i=1}^l{\nu}_i \); l = 14

  

\( \sum_{i=1}^l{\nu}_i \); l = 23

  

-0.002≈0

  

-0.005≈0

C

I

 

C

I

v

3. User organizational performance

1

-1.852

4. Strategic alignment and business impact on user enterprise

1

1.645

2

-1.492

2

0.955

3

-2.132

3

1.875

4

-0.022

4

1.095

5

0.378

5

0.255

6

-0.202

6

-2.245

7

0.118

7

-0.545

8

1.018

8

-0.515

9

0.168

9

0.845

10

2.008

10

0.545

11

2.008

11

-0.075

12

-1.815

13

0.225

14

0.595

15

-0.205

16

0.345

17

1.785

18

-2.615

19

-2.155

  

\( \sum_{i=1}^l{\nu}_i \); l = 11

  

\( \sum_{i=1}^l{\nu}_i \); l = 19

  

-0.002≈0

  

-0.005≈0

Table 2 shows results of the assessment of data availability as well as usefulness of support provided in Geoportal 2. The average score for this category is 4.4 points. Three indicators received the highest weighted average scores (over 6.5 points): data – thematic accuracy (1_1), data – completeness (1_2), data – lineage (1_7). The rest of the scores are equal or below 5.0 points.

As the survey results showed, the PSDI has influence on decision makers and the decision-making process (average score for the category is 6.0 points, Table 3). From among 6 indicators considered for the Decision makers subcategory, 3 of them received the weighted average scores over 6.5 points. In the area of Decision making process 3 indicators out of 14 received weighted average scores over 6.5 points. The lowest scored indicators (weighted average scores below 5.0 points) are better/easier cooperation with different stakeholders (2_19), better/easier cooperation within an organization (2_20) and consideration of constraints and alternatives (2_12). The weighted average scores for the Applications subcategory are between 6.0 and 6.5 points.

The average score for the category of User organizational performance is 4.2 points (Table 4). 3 indicators out of 11 received scores over 5.5 points: duration of procedure (3_1), change of attitude towards some procedures/tasks (3_2), improved procedures (3_3). The rest of the potential effects have very low meaning for land surveyors and spatial planners (weighted average scores are between 2.2 and 4.4 points).

The average score for the Strategic alignment and business impact on user enterprise category is 3.7 points (Table 5). From among 19 effects, 4 received 5.5 points and more: improved knowledge transfer (4_6), optimization of workflow (4_12), possibility of ICT inclusion in tasks (4_18) and ICT impact on efficiency increase of the employees and the whole company (4_19). The other potential effects are not necessarily perceived by the respondents (weighted average scores between 1.8 and 4.2 points).

Total score for the geoportal 2 project

This section presents calculations of the weighted scores for each category (WSc) and total score for the Geoportal 2 project (WSBP). An assessment is conducted in two variants (Tables 8 and 9). The base are weights (W) for categories (Table 1), as well as average scores for categories (ASc) calculations (Tables 2, 3, 4, and 5).
Table 8

Assessment for the Geoportal 2 project (1st variant)

Category ID

C

Wci (%)

ASci (points earned)

Points available

Asci (percentage)

WSci (%)

1

Information and support provided

25

4.4 ± 0.4

10

0.44 ± 0.04

11 ± 1

2

Use process

25

6.0 ± 0.2

10

0.60 ± 0.02

15.0 ± 0.5

3

User organizational performance

25

4.2 ± 0.4

10

0.42 ± 0.04

10.5 ± 1.0

4

Strategic alignment and business impact on user enterprise

25

3.7 ± 0.3

10

0.37 ± 0.03

9.25 ± 0.75

    

WSBP=

45.75 ± 3.25

Table 9

Assessment for the Geoportal 2 project (2nd variant)

Category ID

C

Wci (%)

ASci (points earned)

Points available

Asci (percentage)

WSci (%)

1

Information and support provided

45

4.4 ± 0.4

10

0.44 ± 0.04

19.8 ± 1.8

2

Use process

45

6.0 ± 0.2

10

0.60 ± 0.02

27.0 ± 0.9

3

User organizational performance

5

4.2 ± 0.4

10

0.42 ±0.04

2.1 ± 0.2

4

Strategic alignment and business impact on user enterprise

5

3.7 ± 0.3

10

0.37 ± 0.03

1.85 ± 0.15

    

WSBP=

50.75 ± 3.05

In the first case (Table 8), all categories are of the same importance (25%). The category of Use process received the highest weighted score (15%). The categories of Information and support provided as well as User organizational performance received, in round figures, the same scores (11%). The category of Strategic alignment and business impact on user enterprise received the lowest score (9.25%). The total weighted score for the Geoportal 2 project is 45.75 ± 3.25%. According to the rules for interpretation of overall scores, Geoportal 2 has some potential but some components (datasets, network services, hardware, software) require modification in subsequent projects.

In the second case (Table 9), the European Project INSPIRE and its objectives (European Parliament and the Council 2007; Commission of the European Communities 2009) are taken into account, including the need for collecting information about the use of spatial data services of the infrastructure for spatial information, as well as evidence showing the use of the infrastructure for spatial information by the general public. Therefore, the first two categories are assumed to be the most important (weight of 45% per category). In this case, the category of Use process also received the highest weighted score (27%). The score for the Information and support category is in round figures 20%. The categories of User organizational performance as well as Strategic alignment and business impact on user enterprise received very low scores (2.1% and 1.85% respectively). The total weighted score for the Geoportal 2 project is 50.75 ± 3.05%. According to the rules for interpretation of overall scores Geoportal 2 is good and quite effective, although its components may need some improvements and additional analyses in the future. Although, if accuracy of the averages scores for categories is taken into consideration, awareness of the results’ credibility should be retained and some components of the main access point may require a deeper analysis and some modifications.

Discussion

This ex-post analysis allows for the assessment of the effectiveness of the PSDI adhering to the Geoportal 2 project’s artifacts, i.e. main access point to the PSDI and its applications, datasets and support from the user perspective. The paper presents some preliminary findings about the availability and usefulness of the main Polish access point in professional practice of spatial planners and land surveyors. This is the first use of the multi-criteria method proposed in this study.

Average standard deviation calculations allowed for the evaluation of accuracy of the weighted average scores for each indicator, as well as the average scores for each category (Tables 2-5). The results of calculations of the sums of deviations from averages confirmed the correctness of the averages calculations (Tables 6-7).

The total weighted scores for the Geoportal 2 project (in round figures 46% for the first variant and 51% for the second) indicated that Geoportal 2 has some potential, but some components (datasets, network services, hardware, software, procedures) will require fine-tuning in the future to increase the PSDI effectiveness as indicated by the rules for interpretation of overall scores presented in this paper and the results for the category of Information and support provided as well as the subcategory Applications of the Use process category. Moreover, the variants illustrated importance of the weighting.

The study confirmed that the PSDI is used both by urban planners and surveyors, although the analysis of the scores for the groups suggested that spatial planners value the Geoportal 2 project’s artifacts more and derive more benefits from using them, by comparing the average scores for indicators of the Use process category (Table 3). The PSDI does not have a great effect on the organizational performance of the users’ enterprises oruser enterprise strategy and business goals.

Datasets in the geoportal might be inappropriate for carrying out tasks, considering the scores (Table 2) in the temporal validity, positional accuracy and distribution formats of data categories. Moreover, the surveyor group was of the opinion that the geoportal, which integrates the datasets of selected thematic groups mainly from the central geodetic and cartographic service, is not always able to rival the district or municipality geoportals and websites. Among the different types of user support, email addresses (for spatial planners) and video tutorials (for surveyors) were the most useful ones. The weighted average scores for the applications are between 6.0 and 6.5 points (Table 3), but some technical issues were raised by the interviewees. First of all, Geoportal 2 is perceived as difficult to use because of the layout of certain tools and a lack of intuitiveness. The availability and performance of the applications is also intermittent. The list of factors influencing the potential lack of accessibility includes aspects that are both user-specific (internet access, internet speed, hardware, software, web browsers) and Geoportal 2 administrator-specific (server performance and scale to keep up with growing computing demands).

The multi-criteria method presented in this paper allows for the assessment of the SDI effectiveness from the user perspective by integration of the quantitative non-financial aspects with some qualitative contribution of the SDI, and adds to a variety of methods that have been used for user perspective so far, (e.g. Grus et al. 2008; Nedović-Budić et al. 2004; European Parliament and the Council 2007; Commission of the European Communities 2009):verbal description, case studies, survey, as well as statistical tests. Financial methods, based on financial analysis tools are the most desirable from the investors’ perspective, but include only selected costs and benefits which are defined in the monetary unit. They are considered for SDI projects (e.g. Craglia and Nowak 2006; Bregt 2012; Borzacchiello and Craglia 2013). However, Bregt (2012) underlines that on the one hand, cost-benefit analysis is easy to understand as it translates all aspects into monetary terms, but on the other hand, it is not the right tool for as complex a project as INSPIRE, especially during the phase of the infrastructure implementation. In this paper, it is indicated that qualitative methods, including multi-criteria and strategic analysis, are an interesting option which can be used for SDI projects of different sizes and allows for the integration of financial aspects with non-financial ones.

For the purpose of identifying the indicators for the categories proposed in this study, some concepts from the field of SDI, operational research, management information system and business economics (Hamilton and Chervany 1981; Farbey et al. 1992; Murphy 2002; Nedović-Budić et al. 2008) were incorporated into the method. Moreover, the author’s expert knowledge, gained from the implementation of the Geoportal 2 project, was utilized. Selected indicators allow to capture essence of the concept categories such as effective use, organizational effectiveness, as well as value deriving from IT projects with reference to some basic SDI components such as main access point and its applications, datasets and support provided.

The method is subjective, as it is based on ranking, but this is (Remenyi et al. 2000) the common feature of all multi-criteria methods. The advantages of the method encompass the possibility of modifying the indicators, including adding new ones or selecting the optimum set of measures for a given context or set of circumstances, integrating with the financial category, as well as a wide scope of evaluation. This method is dedicated to organizations responsible for SDI development and monitoring the relationships between the investments in SDI initiatives and the results obtained taking into account usability, usefulness, as well as availability of the infrastructure for users’ enterprises.

Conclusion

The aim of the study was to present a multi-criteria method for ex-post assessment of SDI effectiveness from the users’ perspective, as well as the application of the proposed method in Geoportal 2, the main access point to the Polish Spatial Data Infrastructure’s spatial data and services, as well as two groups of the national geoportal’s users.

The results bring us closer to understanding of the PSDI effectiveness considering the following dimensions: information and support provided, use process, user organizational performance, strategic alignment and business impact on user enterprise. The total scores for the main access point of the PSDI indicated its potential, but the Geoportal 2 project’s artifacts (datasets, network services, hardware, software) will require fine-tuning including improvements and additional analyses in the future to increase the PSDI effectiveness. The PSDI and Geoportal 2 are considered by surveyors to bea source of information supplementary to documentation from the geodetic and cartographic service centers or field reconnaissance. They value the Geoportal 2 project’s products and use them for land surveying work activities. Spatial planners perceive Geoportal 2 as the data source for 14 different tasks.

A literature overview confirms there are examples of application of different qualitative methods for assessment of SDIs which come from different scientific fields, including operational research, management and economy. Geudens et al. (2009) present a new multi-criteria method dedicated to SDI which is for ex-ante assessment of new SDI policy strategies. There is also evidence of use of the Balanced Scorecard method (BSC) for SDI in the literature (Toomanian et al. 2011). Generally, BSC is the universal method for assessment of organizational effects. Thus, the BSC (Lech 2005) focuses only on one aspect of IT impact on the organization responsible for IT project implementation – supporting the strategic goals. A concept of a qualitative method introduced in this paper is IT-focused and considered for the SDI business project approach, and adds to a variety of viewpoints and methods identified for the Multi-View SDI Assessment Framework. It covers all three purposes of the assessment linked with the SDI assessment framework: accountability, knowledge and development. It is for ex-post and monitoring assessment of using the SDI business project’s artifacts – outputs, e.g. main access point and its applications, datasets, network services, metadata, hardware, which constitute the SDI components. It also develops the concept of the SDI effectiveness and contributes to the user perspective described by Nedović-Budić et al. (2008), by references to aspects such as usefulness and usability, as well as information and organizational effectiveness. The multi-criterion method proposed in this study widens the typology of methods used for SDI assessment from a user perspective and helps to evaluate the mentioned above aspect using a wide range of quantitative non-financial and also qualitative aspects. The proposed scope of effectiveness assessment and indicators make also references to the aspects of system effectiveness and business value from information technology, as well as derive from other scientific fields such as business economics and management information system.

Application of the method requires a survey instrument, which should be generally accessible. Popularization of the survey among communities interested in using the SDI is recommended. Survey forms could be presented on the web pages connected with the SDI data and service source access points.

The results of the SDI assessment may serve as a basis for improvements of components of the SDI in the future, as well as identifying goals for the SDI development and the following SDI projects carried out by the Polish Head Office of Geodesy and Cartography. Including respondents’ recommendations concerning the main access point to the PSDI – Geoportal 2 – such as (1) better spatial data quality, (2) user-friendly application layout, (3) intuitive access to application toolbars as well as (4) high server performance and appropriate scale to keep up with growing computing demands will certainly increase the effectiveness of the PSDI and the Geoportal 2.

References

  1. Bannister F (2001) Dismantling the silos: extracting new value from IT investments in public administration. Inf Syst J 11:65–84CrossRefGoogle Scholar
  2. Borzacchiello MT, Craglia M (2013) Estimating benefits of spatial data infrastructures: a case study on e-Cadastres. Comput Environ Urban 41:276–288CrossRefGoogle Scholar
  3. Bregt A (2012) Spatial Data Infrastructures. Cost-Benefit Analysis in Perspective. Costs and Benefits of Implementing the INSPIRE Directive Workshop. JRCGoogle Scholar
  4. Chan TO, Feeney ME, Rajabifard A, Williamson I (2001) The dynamic nature of spatial data infrastructure: a method of descriptive classification. Geomatica 55(1):451–458Google Scholar
  5. Commission of the European Communities (2009) Decision of 5 June 2009 implementing Directive 2007/2/EC of the European Parliament and of the Council as regards monitoring and reporting. Official Journal of the European Union L148, Luxembourg: Publications Office of the European UnionGoogle Scholar
  6. Craglia M, Campagna M (2010) Advanced regional SDIs in Europe: comparative cost-benefit evaluation and impact assessment perspectives. International Journal of Spatial Data Infrastructures Research 5:145–167Google Scholar
  7. Craglia M, Nowak J (2006) Report of International Workshop on Spatial Data Infrastructures: Cost-Benefit / Return on Investment. 12–13 January 2006. Ispra, Italy. European Commission Joint Research Centre, Institute for Environment and SustainabilityGoogle Scholar
  8. de By RA, Lemmens R, Morales J (2009) A skeleton design theory for spatial data infrastructure. Methodical construction of SDI nodes and SDI networks. Earth Sci Inform 2:299–313CrossRefGoogle Scholar
  9. European Parliament and the Council (2007) Directive 2007/2/EC of the European Parliament and of the council of 14 March 2007 establishing an Infrastructure for Spatial Information in the European Community (INSPIRE). Official Journal of the European Union L108(50). Publications Office of the European UnionGoogle Scholar
  10. Farbey B, Land F, Targett D (1992) Evaluating investments in IT. J Inform Technol 7(2):109–122CrossRefGoogle Scholar
  11. Geudens T, Macharis C, Crompvoets J, Plastria F (2009) Assessing spatial data infrastructure policy strategies using the multi-actor multi-criteria analysis. International Journal of Spatial Data Infrastructures Research 4:265–297Google Scholar
  12. Giff GA, Crompvoets J (2008) Performance indicators a tool to support spatial data infrastructure assessment. Comput Environ Urban 32(5):365–376CrossRefGoogle Scholar
  13. Grus L, Crompvoets J, Bregt AK (2007) Multi-view SDI assessment framework. International Journal of Spatial Data Infrastructures Research 2:33–53Google Scholar
  14. Grus L, Crompvoets J, Bregt AK (2008) Theoretical introduction to the multi-view framework to assess SDIs. In: Crompvoets J, Rajabifard A, van Loenen B, Delgado Fernandez T (eds) A multi-view framework to assess spatial data infrastructures. The Melbourne University Press, Melbourne, pp 93–113Google Scholar
  15. Grus L, Castelein W, Crompvoets J, Overduin T, van Loenen B, van Groenestijn A, Rajabifard A, Bregt AK (2011) An assessment view to evaluate whether spatial data infrastructures meet their goals. Comput Environ Urban 35(3):217–229CrossRefGoogle Scholar
  16. Hamilton S, Chervany NL (1981) Evaluating information system effectiveness — part I: comparing evaluation approaches. MIS Quart 5:55–69CrossRefGoogle Scholar
  17. Heffron F (1989) Organizational theory and public organizations: the political connection. Englewood CliffsGoogle Scholar
  18. Hjelmager J, Moellering H, Cooper A, Delgado T, Rajabifard A, Rapant P, Danko D, Huet M, Laurent D, Aalders H, Iwaniak A, Abad P, Düren U, Martynenko A (2008) An initial formal model for spatial data infrastructures. Int J Geogr Inf Sci 22(11–12):1295–1309CrossRefGoogle Scholar
  19. Innerebner M, Costa A, Chuprikova E, Monsorno R, Ventura B (2016) Organizing earth observation data inside a spatial datainfrastructure. Earth Sci Inform. doi:10.1007/s12145-016-0276-0 Google Scholar
  20. Kok B, van Loenen B (2005) How to assess the success of National Spatial Data Infrastructures? Comp Environ Urban 29(6):699–717CrossRefGoogle Scholar
  21. Lance KT, Georgiadou Y, Bregt A (2006) Understanding how and why practicioners evaluate SDI performance. International Journal of Spatial Data Infrastructure Research 1:65–104Google Scholar
  22. Lech P (2005) Evaluation Methods’ Matrix – A Tool for Customized IT Investment Evaluation. Proceedings of the 12th European Conference on Information Technology Evaluation: 297–306Google Scholar
  23. Murphy T (2002) Business value from technology. A practical guide for today's executive, John Wiley & Sons, Inc, HobokenGoogle Scholar
  24. Nedović-Budić Z, Feeney MEF, Rajabifard A, Williamson I (2004) Are SDIs serving the needs of local planning ? Case study of Victoria, Australia and Illinois, USA. Comp Environ Urban 28:329–351CrossRefGoogle Scholar
  25. Nedović-Budić Z, Pinto JK, Budhathoki NR (2008) SDI effectiveness from the user perspective. In: Crompvoets J, Rajabifard A, van Loenen B, Delgado Fernandez T (eds) A multi-view framework to assess spatial data infrastructures. The Melbourne University Press, Melbourne, pp 273–303Google Scholar
  26. Parker M, Benson J (1988) Information economics. Prentice Hall, Upper Saddle RiverGoogle Scholar
  27. PMBOK (2000) A guide of the Project Management body of knowledge, Project Management Institute, USAGoogle Scholar
  28. Polish Head Office of Geodesy and Cartography (2009) Feasibility study of the geoportal 2 projectGoogle Scholar
  29. Polish Head Office of Geodesy and Cartography (2010) Architecture of the geoportal 2 information systemGoogle Scholar
  30. Productivity Commission (2013) On efficiency and effectiveness: some definitions. Staff Research Note, CanberraGoogle Scholar
  31. Rajabifard A, Feeney MEF, Williamson IP (2002) Future directions for SDI development. Int J Appl Earth Obs 4(1):11–22CrossRefGoogle Scholar
  32. Remenyi D, Money A, Sherwood-Smith M (2000) The effective measurement and management of IT costs and benefits. Butterworth-Heinemann, OxfordGoogle Scholar
  33. Renkema T, Berghout E (1997) Methodologies for information systems investment evaluation at the proposal stage: a comparative review. Inform Software Tech 39:1–13CrossRefGoogle Scholar
  34. Steudler D, Rajabifard A, Williamson I (2008) Evaluation and performance indicators to assess spatial data infrastructure initiatives. In: Crompvoets J, Rajabifard A, van Loenen B, Delgado Fernandez T (eds) A multi-view framework to assess spatial data infrastructures. The Melbourne University Press, Melbourne, pp 193–210Google Scholar
  35. Toomanian A, Mansourian A, Harrie L, Ryden A (2011) Using balanced scorecard for evaluation of spatial data infrastructures: a Swedish case study in accordance with INSPIRE. International Journal of Spatial Data Infrastructures Research 6:311–343Google Scholar
  36. Vandenbrucke D, Dessers E, Crompvoets J, Bregt AK, van Orshoven J (2013) A methodology to assess the performance of spatial data infrastructures in the context of work process. Comput Environ Urban 38:58–66CrossRefGoogle Scholar
  37. Wortman KC (1994) Developing standards for a National Spatial Data Infrastructure. Cartogr Geogr Inform 21(3):132–135CrossRefGoogle Scholar
  38. Zwirowicz-Rutkowska A (2014) A business project approach to assess spatial data infrastructures. Conference proceedings of the 14th Geoconference on informatics. Geoinformatics and Remote sensing 3:413–421Google Scholar
  39. Zwirowicz-Rutkowska A (2016) Evaluating spatial data infrastructure as a data source for land surveying. Journal of Surveying Engineering-ASCE. doi:10.1061/(ASCE)SU.1943-5428.0000185 Google Scholar

Copyright information

© The Author(s) 2017

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.

Authors and Affiliations

  1. 1.University of Warmia and Mazury in OlsztynOlsztynPoland

Personalised recommendations