A systematic approach for the estimation of software risk and cost using esrcTool

Abstract

Software risk evaluation is a process for identifying, analysing, and developing mitigation strategies for risk in a software intensive system while it is in development. This paper presents a systematic approach for the estimation of software risk and cost using esrcTool. This tool is based on software risk assessment and estimation model. In this model function point approach is employed as an input variable to estimate the source of uncertainty, i.e. measurement error, model error, and assumption error. To show the validity of our tool, we have considered the project developed by our students.

1 Introduction

Software risk evaluation (SRE) is a process for identifying, analyzing, and developing mitigation strategies for risks in a software-intensive system while it is in development. SRE process has been in evolutionary development at the Software Engineering Institute (SEI) since 1992; and it has been used on over 50 Department of Defence and civil (Federal and State) contractors and program offices. There are certain questions that are related to risk management like risk prioritization and analysis etc. It is systematic and continuous processes that can be best described by the SEI risk management paradigms which includes: (a) identify (b) analyze (c) plan (d) track (e) control and (f) communication [1]. The practice of risk management involves two primary steps each with three subsidiary steps, as shown in Fig. 1. The first primary step of risk management is risk assessment and it involves: (a) risk identification, (b) risk analysis, (c) and risk prioritization. The secondary step of risk management includes risk control and it involves: (a) risk management planning, (b) risk resolution, (c) and risk monitoring [2–4].

Fig. 1

Steps involved in the risk management

Risk assessment begins with team training. The team meets prior to the risk assessment for team building and training in the details of risk management paradigm and risk assessment process. This training includes instruction in the risk management mechanism to be applied during the assessment as well as practice exercises using the mechanism [5, 6]. Brief description about primary and secondary steps of risk management is given below:

• 1. Risk identification produces lists of the projects-specific risk items likely to compromise a project success. A typical risk identification technique includes examination of decision drivers, assumption analysis, and checklist.

• 2. Risk analysis assesses the loss probability and loss magnitude for each identified risk item and it access compound risk in the risk item interactions. Typical technique include performance models, cost models, network analysis etc.

• 3. Risk prioritization produces a ranked ordering of the risk items identified and analyzed. Typical techniques includes risk exposure analysis, risk reduction leverage (particularly involve cost-benefit analysis) etc.

• 4. Risk management planning helps prepare you to address each risk item including the coordination of individual risk item plans with each other and with the overall project plan. Typical technique includes checklist for risk resolution technique, cost benefit analysis, and standard risk management plan outlines, form and elements.

• 5. Risk resolution produces a situation in which the risk items are eliminated or otherwise resolved (for example risk avoidance via relaxation of requirements). Typical technique includes prototypes simulation, benchmark, mission analysis, and design to cost approach.

• 6. Risk monitoring involves tracking the project’s progress toward resolving its risk items and taking corrective action where appropriate. Typical technique includes milestone tracking and a top ten risk item list that is highlighted at each weekly and monthly [7].

The objective of this paper is to present a systematic approach to estimate the software risk and cost using esrcTool. This tool is based on software risk assessment and estimation model (SRAEM) [8] which is used to predict the possible results of software projects with good accuracy. SRAEM not only assess the risk but it also estimates the risk. In SRAEM, function point (FP) is used to estimate the measurement error, model error, and assumption error. Function point is an important software metrics which is used to calculate the approximate LOC, cost and effort of software [5, 9, 10].

The paper is organized as follows: In Sect. 2 we present the background and the related work of risk assessment and estimation model. Section 3 presents the method for the computation of FP. In Sect. 4, we present the systematic approach for the estimation of software risk and cost using esrcTool. An experimental work is carried out in Sect. 5; and finally we conclude the paper in Sect. 6.

2 Background and related work

This section presents the background and related work of software risk assessment and estimation models. Researchers, scientist and academician in the field of software engineering have developed a lot of models/tools according to their need and requirements. Gupta and Sadiq [8] have developed a SRAEM to predict the possible results of software projects with good accuracy. SRAEM model not only assess the risk but it also estimate the risk. In this model, the risk is estimated using risk exposure and software metrics of risk management. This metric is based on mission critical requirements stability risk metrics (MCRSRM) [11]. This software metrics is used when there are changes in requirements such as addition, subtraction, or deletion. The esrcTool gives the incremental risk for every phase and also the total cumulative risk as the software progress from phase to phase.

Gupta et al. [12] developed a software estimation tool based on software engineering metrics model. In literature, there are a few published models that evaluate the risk of software projects. In [13] Kashlaf and Hashim developed a model and prototype tool to manage software risks. Soft risk prototype is a tool prototype to manage software development risks. Java language has been chosen as development language of the prototype. Another model named Software engineering risk model (SERIM) focuses on three risk elements: (i) technical risk, (ii) cost risk, and (iii) schedule risk. This model does not take into account of the software complexity issues, which plays an important role in determining the risk for the software projects. It also does not account for issues related to requirements. In the series of risk assessment models there is another model called software risk assessment model (SRAM) [14]. This model considers the nine critical risk elements (i) complexity of the software (ii) staff involved in the projects (iii) targeted reliability (iv) product requirement (v) method of estimation (vi) method of monitoring (vii) development process adopted (viii) usability of software (ix) tools. The above models do not include the sources of uncertainty, i.e. measurement error, model error and assumption errors. Existing models have considered the prioritization as a single step of risk assessment but does not specify how prioritization would be done. In [15] we proposed the architecture of an esrcTool to estimate the software risk and cost. On the basis of systematic review of software risk assessment and estimation model [16], which is an extension of the work of Georgieve et al. [17], we identify 11 primary studies that are related to software risk and cost estimation and its brief description is given in Table 1. The results of previous studies indicate that SRAEM [8] and SRAEP using model based approach [18] is the latest risk assessment model in the literature [16, 17].

Table 1 Summary of various risk assessment methods [16]

3 Method for the computation of FP

At the beginning of the 1970s, researchers at IBM initiated studies aimed at determining what critical variables were involved in programming productivity. Instead of considering code volume or complexity, they discovered that a system would be better evaluated by analyzing the functions executed by programs and mapping pertinent questions to estimating and evaluating software’s development productivity in heterogeneous environments [19]. Function Point is a well known established method to estimate the size of software system and software projects. Originally, the method was used in the early phases of the waterfall model such that the implementation effort could be estimated on the basis of input and output behaviour as defined in the functional documentation.

As the size and the complexity of software increases, it becomes increasingly important to develop high quality software and cost effectively within a specified period. In order to achieve this goal, the entire software development processes need to be managed based on an effective plan. The subjects of estimation in the area of software development are size, effort invested, development time, technology used and quality.

Function Point is a measure of software size that uses logical functional terms business owners and users more readily understand. Albrecht’s model of functional specifications requires the identification of five types of components, namely input, output and inquiry elementary processes and logical internal elementary processes and logical internal and external interface files The actual calculation process itself is accomplished in three stages: (I) determine the unadjusted function point (UFP), (II) value adjustment factor (VAF) and (III) Adjusted function points(AFP).The unadjusted function points includes: (a) Data function and (b) Transactional functions.

The data function includes:

• 3.1 Internal logical file

• 3.2 External interface file.

The transactional functions are classified in the following manner:

• 3.3 External input.

• 3.4 External output.

• 3.5 External enquiry.

3.1 Internal logical file (ILF)

ILF are groups of logically related data or control information maintained by the application itself. For instance, the file that stores the customer data of a company is and ILF for the customer database system since such a system is responsible for customer maintenance.

3.2 External interface file (EIF)

EIF are groups of logically related data or control information whose maintenance is under the responsibility of another application. For instance, the file that stores employee data of a company is an EIF for the customer database system, assuming that it accesses employee data whose actual maintenance is accomplished by the employee database system.

3.3 External input (EI)

EI are elementary processes that involve data or control information that are input at the boundary of the application with the main objective of doing ILF maintenance, for instance, updating the personal data of customer of an organization.

3.4 External output (EO)

EO are elementary processes that send control information or calculated data to the end user or to other applications, for instance, a report that summarizes sales volume per month for a particular customer of a company.

3.5 External inquiry (EQ)

EQ are elementary processes that send control information or uncalculated data to the end user or to other applications. For instance, a personal data query operation for an employee of a company

Once we have identified EI, EO, EQ, ILF, and EIF then each function presented must be classified according to its relative functional complexity as low, average or high. Calculating the VAF, is an earmark of the general functionality provided to the user. The VAF is derived from the sum of the degree of influence (DI) of the 14 general system characteristics. The DI of each one of these characteristics ranges from 0 to 5 as follows:

• (i) 0––no influence;

• (ii) 1––incidental influence;

• (iii) 2––moderate influence;

• (iv) 3––average influence;

• (v) 4––significant influence; and

• (vi) 5––strong influence.

The general characteristics (F_i) of a system are: (i) data communications; (ii) distributed data processing; (iii) performance; (iv) heavily used configuration; (v) transaction rate;(vi) online data entry; (vii) end user efficiency (viii) online update(ix) complex processing; (x) reusability; (xi) installation ease; (xii) operational ease; (xiii) multiple sites; (xiv) facilitate change. The third and the last stage is the final calculation of the function points. With the help of the following equation we can get the total points of an application.

$$ {\text{AFP}} = {\text{UFP}} \times {\text{VAF}}$$

where AFP adjusted function points; UFP unadjusted function points; and VAF value adjustment factor [20, 21].

Function points are computed by completing the following Table 2. Five information domain characteristics are determined and counts are provided in appropriate table location [22].

Table 2 FP computation

Organization that use FP methods develop criteria for determining whether a particular entity is simple, average, or complex. To compute the AFP the following relationship is used.

$$ {\text{AFP}} = {\text{UFP}} \times \left[ {0. 6 5+ 0.0 1\times \sum {\left( {{\text{F}}_{\text{i}} } \right)} } \right] $$

(1)

where F_i is the complexity adjustment value, i = 1–14.

For example, it is given that EI = 5, EO = 2, EQ = 4, ILF = 2 and EIF = 1 and the complexity of the project is average then the UFP would be calculated as follows (see Table 3).

Table 3 Results

If we assume that all general systematic characteristics are absolutely essential, then FP in this case would be 98.55

4 Systematic approach

This section presents a systematic approach for the estimation of software risk and cost using esrcTool l [15, 23]. Systematic means to characterize by order and planning. Therefore, esrcTool first extract the source code of the program/software for the computation of the FP because it is used as an input to the measurements error, model error, and assumption error. In order to estimate the risk and cost of the software, we have implemented this tool in C language. The architecture of the esrcTool is given in Fig. 2.

Fig. 2

Architecture of esrcTool (adopted from [23])

The systematic approach for the estimation of software risk and cost using esrcTool involves the following steps: (a) estimation of the risk, (b) cost estimation. The detailed descriptions about these steps are given in the following sub-section.

4.1 Estimation of the risk

There are three dimensions of software risk i.e. technical risk, organization and environmental risk [16]. Each software models have some weaknesses and also have some advantages [22]. A technical report on the software risk evaluation method is available in [1]. The esrcTool estimates the risk on the basis of measurement error, model error, and assumption error.

4.1.1 Measurement error

This error occurs if some of the input variables in a model have inherent accuracy limitations. Kemerer [24] argue that, function points are assumed to be at least 12 % inaccurate. Thus, if we estimate a product size of 1,000 function points, measurement error could mean that the real size is anywhere between 880 and 1,120.

4.1.2 Model error

Factors that affect error but are not included explicitly in the model contribute to the model error. For example, 0.5 person-days per function point is usually obtained from results observed for recalled from previous projects. It is unlikely that any future projects will achieve the same ratio, but the model is expected to all right on average. If you base a model on past project data, you should calculate the associated inaccuracy by using the mean magnitude relative error. Thus, if you have estimation model with an inherent 20 % inaccuracy and your product is 1,300 function points in size; your estimate is likely to be between 208 and 312 person days.

4.1.3 Assumption error

This error occurs when we make incorrect assumptions about a model’s input parameters. For example, your assessment that a product size is 1,300 function point rests on the assumption that you have correctly identified the customer requirements. If you can identify your assumptions, you can investigate the effect of their being invalid by assessing both the probability that an assumption is incorrect and the resulting impact on the estimate.

This is the form of risk analysis. For example, if you believe that there is a 0.3 probability that the requirement complexity has been underestimated and, if it has, you estimate another 390 function point. At this point the concept of risk exposure is used to calculate the effective current cost of a risk and can be used to prioritize risk that requires countermeasure. Mathematically it can be written as:

$$ {\text{Probability}}\,{\text{of}}\,{\text{risk}}\,{\text{occurring}} \times {\text{Total}}\,{\text{loss}}\,{\text{if}}\,{\text{risk}}\,{\text{occur}} $$

(2)

total loss can be defined as E2 − E1. Where E1 is the effort, if the original assumption is true and E2 is the effort if the alternative assumption is true. Suppose E1 = 540 person days and E2 = (1,300 + 100) × 0.5 = 700 person days, then risk exposure = (700 − 540) × 0.3 = 48 person days.

The esrcTool also includes the MCRSRM in order to compute the risk when there are some changes in the requirements (addition, modification, or deletion). Therefore, total risk can be computed as [23]:

$$ \left[ {{\text{b}}/{\text{a}}} \right]_{{{\text{i}} = 1\,{\text{to}}\,{\text{n}}}} + {\text{K}}\left[ {{\text{A}}\left[ {{\text{c}}/{\text{d}}} \right]_{\text{i}} + {\text{B}}\left[ {{\text{d}}/{\text{b}}} \right]_{\text{i}} + {\text{G}}\left[ {{\text{e}}/{\text{b}}} \right]_{\text{i}} } \right] $$

(3)

where:

• (i) [b/a]_i = (number of mission critical requirements)/(total number of requirements) at the input of phase number i,

• (ii) K_i is the penalty for adding, modifying or deleting of requirements during phase number i,

• (iii) a = total number of requirements,

• (iv) b = total number of mission critical requirements (MCR),

• (v) c = number of MCR added during phase i,

• (vi) d = number of MCR modified during phase i,

• (vii) e = number of MCR deleted during phase i.

On the basis of the following parameters during phase 1, the risk at the input is 40/100 and the added risk during phase 1 is 42/40 due to adding, modifying and deletion of MCR. Where, a = 100, b = 40, c = 5, d = number of MCR modified during phase 1 = 10 (3 which were downgraded from MCR to just requirements, and the remaining 7 are still MCR, and e = 7.

It can be seen that esrcTool gives the incremental risk for every phase and also the total cumulative risk as the project progresses from phase to phase. Assumption error helps you to estimate the risk exposure. Now the next step is how to prioritize the risk: Suppose in a software project, we identified three different types of risk i.e. products recall situation, significant product rejection, and competitive strike. The information about probability of risks occurring and the total loss if it occurs are given in the Table 4.

Table 4 Information about probability of occurring and total loss, if it occurs

The rank of risk is estimated using risk exposure and the value of the highest risk exposure indicate the most serious risk. Table 5 contains the calculated values of risk exposure and the ranking of risk.

Table 5 Prioritization of risk

In Table 5, Competitive strike contains the highest value of risk exposure i.e. 2,500, so it has the first priority. Similarly, product recall situation and significant product rejection have second and third priority respectively. Since risk exposure is not absolute but relative. There are several ways to compare different exposures with each other. One ways is to compare the exposure of a single event before and after managing the risk. We need a simple measure to assess risk reduction. So risk reduction leverage (RRL) is another quantitative means of assessing how risks are being managed. Mathematically we can write the RRL as:

$$ \left( {{\text{Risk}}\,{\text{exposure}}\,{\text{before}} - {\text{Risk}}\,{\text{exposure}}\,{\text{after}}} \right)/{\text{Cost}}\,{\text{of}}\,{\text{risk}}\,{\text{reduction}} $$

(4)

4.2 Cost estimation

International Software Benchmarking Standards Group (ISBSG) is an international group of representatives from international metrics organizations who collect project data from countries like, India, Hong Kong Germany, Japan, and USA. ISBSG Release 6 Report provides the cost value for the software projects. Cost data is derived from 56 projects representing a broad cross section of the software industry. After going through these software projects, the ISBSG conclude that median cost to develop a function point is $US 716, and the average cost is $ US 849 per function point. For more information about the ISBSG please visit: www.ISBSG.org.au

5 Experimental work

This section presents the experimental work. In this paper, we have considered the projects developed by the students of Master of Technology (M.Tech.) of Computer Science and Engineering. The first project that we have considered is “A Mini Software for Numerical Integration (MSNI)”. This Software basically calculates the values of the given Integrand after applying all the existing algorithms of the numerical computations like Simpson’s 1/3 rule, Simpson’s 3/8 rule and so on; and some proposed algorithm. Software requirements elicitation is a process which is used to identify the high level objective of an organization [25–27]. In MSNI, we have collected 21 different requirements and the prioritization of these requirements is shown in Fig. 3. For the computation of software risk and cost, we implement esrcTool in C language; and the GUI of the proposed tool is given in Fig. 4.

Fig. 3

Prioritization of requirements

Fig. 4

Snapshot of the GUI of the esrcTool [23]

In order to estimate the function point of the MSNI, we used the esrcTool as well as the tool proposed by [12]. The value of function point of MSNI found to be 50. According to the measurement error the actual size of the function point varies from 44 to 56. To find out the value of model error, we have assumed that the 0.2 person-days per function point. No estimation model can include all factors that affect the effort required to produce a software product. Suppose, we have an estimation model with an inherent 20 % inaccuracy and our product is of 50 function points in size, our estimation is likely to be between 8 and 12 person days.

The assumption error occurs when we have some incorrect assumption about the models input parameter. Our assumption is that the product size is of 50 function points rest on the assumption that we have correctly identified all the requirements. If we can identify our assumption, we can investigate the effect of their invalid by assessing both the probability hat an assumption is incorrect and the resulting impact on the estimate. This is the called the risk analysis. If we assume that there are 0.4 probabilities that the requirement complexity has been underestimated, so we estimate another 2 function point. We can estimate the risk exposure from the following formula: Risk Exposure = E2 − E1, where E1 is the effort if the original assumption is true, and E2 is the effort if the alternative assumption is true and P2 is the probability that the alternative assumption is true. So in our case E1 = 10 person days, E2 = 50 × 0.2 + 2 × 0.2 = 10.4 = 11(approximately). The risk exposure = 11 − 10 = 1 person days. We have summarized the results of esrcTool in Table 6.

Table 6 Results from esrcTool

6 Conclusion and future work

In this paper we present a systematic approach to estimate the software risk and cost using esrcTool. We have implemented esrcTool using C language. From the proposed tool, it is easy to estimate the risk in the software and also to estimate the cost of the software. The cost of the software depends on the value of the function point. In this paper we have employed the function point approach as an input parameter into the esrcTool. We have applied to the proposed tool on the MSNI; and on nine other projects that are based on software engineering and computer graphics. From the proposed tool, it is easy to find out the cost of software; and estimate the risk of those software’s projects that were designed and developed by our graduate and post graduate students. Software risk and cost analysis is the part of non functional requirements (NFR). Therefore, such type of analysis helps to improve the understanding level of stakeholders during retirements elicitation phase. Future research agenda includes the following:

• (i) To apply the esrcTool on real projects because projects developed by students have some kind of errors.

• (ii) To develop a fuzzy based model for the computation of esrcTool.

• (iii) To elicit and prioritize the software requirements using analytic hierarchy process (AHP) and quality function deployment.