Background

Modern advances in information technology (IT) have completely transformed the face of the world, and IT-based services have significantly improved healthcare services [1, 2]. Among those, HISs are one of the most commonly used health information systems [3]. Despite the potential benefits, HISs are neither widely used by healthcare providers nor are they readily accepted by users where they are launched [4]. Given the significant failure rate of the HIS, efforts are required to be made to enhance its currently-slow popularity by meeting the implementation challenges of this system [5].

Failing to properly identify technical requirements for the HIS and its integrity are implementation challenges of this system [6]. Designing a framework for determining the technical requirements and accordingly evaluating the system is therefore crucial. Technical dimensions of information systems such as architecture, communication service and response time play a key role in the failure rates of these systems [7]. The technical factors have been addressed as the second leading or even the main obstacle to effectively implementing the HIS [1, 8]. Meeting technical requirements ensures information security and confidentiality as well as the successful communication of the system with other systems [9]. Determining the technical requirements assists network designers in identifying technical features and their supporting software version and also in selecting hardware platforms [10].

Given the significant relationships between technical factors and success rates of the HIS [11], establishing appropriate evaluative criteria is as crucial for overcoming technical obstacles to the system implementation [12] as accurately assessing the system is essential for its successful implementation [13].

The HIS evaluation and technical requirements have been addressed in literature in two domains or in a general manner in a single setting using a small sample and a maximum of 7 items [14,15,16,17,18]. Moreover, hardware, software, networking and technical support have been mentioned as the technical challenges facing the HIS development [19, 20]. The Iranian Ministry of Health and Medical Education has defined technical requirements for the HIS in two domains of communication services and security services [21]. A review of literature suggests the very limited number of requirements defined using a small sample mostly involve a small section of hospitals. Library research was also found to have failed to determine comprehensive requirements. The only detailed list of requirements defined by the Iranian Ministry of Health and Medical Education can be developed in terms of the defined domains and items.

Regarding the program of Iranian Ministry of Health and Medical Education for the application of IT services in the healthcare sector and development of electronic health records (EHRs) [22] and considering HISs as good examples of EHRs [23], implementation principles and the technical requirements of these systems need to be continuously assessed and revised [24]. The present study therefore proposed a comprehensive framework for defining technical requirements for the HIS and evaluating this system accordingly. These requirements are recommended to be employed by hospital executive teams and HIS developers.

Methods

The present study was conducted in two parts, i.e. designing technical requirements for the HIS and its evaluation based on these requirements.

First part: designing technical HIS requirements

This part determined technical requirements for the HIS in three stages: Focus Group Discussion (FGD), Modified Delphi Technique and Classical Delphi Technique.

Focus group discussion

Ten experts with at least a master’s degree in software, networking or IT and 7 years of work experience in IT departments of hospitals or medical science universities were first selected from faculty members and the staff. Before holding FGD, a panel of experts was presented with a primary checklist extracted from the comprehensive outline published by the Iranian Ministry of Health and Medical Education regarding HIS assessment criteria [21].

The challenges and technical obstacles to implementing the HIS were discussed in four 4-h sessions. The FGD was held to identify the major technical requirements from expert perspectives and determine all the technical and implementation problems that had ever faced the experts. At this stage, the experts freely exchanged ideas and discussed technical requirements for the HIS through brainstorming. The expert comments were recorded and then transcribed during the meetings. A list of technical requirements was ultimately developed and they were classified into four domains, i.e. communication service, system architecture, security service and system response time according to the expert comments. The primary questionnaire was then designed as the framework of technical requirements for the HIS, which included the study introduction and objectives and a list of technical requirements for the HIS with an open-ended question for including future expert comments.

Modified Delphi technique

The Modified Delphi Technique was employed at this stage to reach consensus on the requirements by going through a multi-stage process. As an open-ended round Delphi guided by focus groups or one-to-one interviews, this technique is generally used as a Group Delphi Technique to reach consensus using expert comments and structured questionnaires [25]. Experts present at this stage were those who participated in the FGD. At the end of the focus group, the initial technical requirements for the HIS were individually scored by the experts on a five-point Likert scale ranging from 0:strongly disagree to 4:strongly agree. The requirements receiving a total mean score of over 3 were approved, those with a total score of below 2 were eliminated and those with a mean score of 2–3 were reassessed by the experts to be either approved or removed from the questionnaire. The experts were encouraged to freely express their ideas and propose potential technical requirements for the HIS. They unanimously finalized the HIS technical questionnaire after holding four focus group rounds. Four groups of technical requirements for the HIS were ultimately accepted as a 72-item questionnaire with a mean score of 3.4.

Classical Delphi technique

At this stage, the 72-item questionnaire approved in the Modified Delphi Technique was distributed among 40 IT experts and IT managers of hospitals and medical science universities across the country with at least a master’s degree and a minimum of 5 years of work experience in the HIS field. This questionnaire included demographic information, 72 closed-ended items and an open-ended question to explore the possibility of including additional technical requirements (Additional file 1). The respondents scored the items on a five-point Likert scale ranging from 0: strongly disagree to 4: strongly agree. At all rounds of the Delphi technique, the participants were anonymously provided with feedback. To meet all the Delphi requirements, including anonymity and feedback, research collaborators were introduced to the study hospitals as another focal point. They were in charge of briefing the IT experts and managers on the study objectives and providing them with feedback on the data obtained at individual rounds. Thirty eight of the 40 questionnaires distributed were completed. The mean scores of the individual requirements calculated in SPSS Statistics for Windows, version 18.0 (SPSS Inc., Chicago, Ill., USA) were employed in statistical analyses. The Classical Delphi Technique was performed in two rounds. The ultimately-approved 73-item questionnaire included one additional item and four domains of technical requirements for the HIS with a mean score of 3.6.

Second part: evaluating HISs

After inquiring of the Iranian Ministry of Health and Medical Education about the number of companies supplying the HIS, the software was found to be provided by 19 software suppliers throughout the country, 16 of which consented to have their HIS evaluated and provided a list of hospitals in which their system was being run for at least 5 years. A total of 16 hospitals, each corresponding to one of the suppliers, were then randomly selected to evaluate the HIS program. Running all the software modules was the only inclusion criterion for selecting a hospital. After completing the HIS technical checklists containing Yes/No items designed in the first part through individually asking from the IT users and experts working at the IT units of the selected hospitals as well as observing the implemented system, the data collected were analyzed in SPSS-18. The hospital HIS received a score of 1 if it met the checklist criteria; otherwise, it received a score of 0. The system performance was evaluated in individual domains as a score in percentage, with 0–20% denoting a very poor performance, 20–40% a poor performance, 40–60% an average performance, 60–80% a good performance and 80–100% a very good performance.

Ethical considerations

Before beginning the FGD, all the participants signed informed consent forms, in which they were briefed on the study objectives and their voluntary participation in the FGD and confidentiality of their information were ensured. Before completing the checklist in the second part of the study, the participants were briefed on the study objectives. Their voluntary participation and confidentiality of their information were also ensured.

Results

In the Classic Delphi stage, 76.3% of the experts were male and 23.7% female. The participants were also 28–52 years old and had a mean age of 34.7 ± 6.3 years. They also had a mean work experience of 9.9 ± 5.3 years and a mean HIS work experience of 6.6 ± 1.7 years, whereas 63.2% had a bachelor’s degree.

In the first round of the Classical Delphi Technique, the experts approved all the requirements associated with communication service, system architecture, security service and system response time. A requirement associated with system architecture, which did not receive the minimum score in the first round, was approved after being reassessed along with the newly-proposed requirements in the second round of the Classical Delphi Technique. The 73-item list of HIS technical requirements was ultimately confirmed with a mean score of 3.6 in the four domains. These requirements were met in 68.9% of the HISs run in the study hospitals. Tables 1, 2, 3, 4 and 5 present the results of the Delphi technique and evaluation of HISs technical requirements.

Table 1 The mean score of the technical requirements and the HIS evaluation results
Full size table
Table 2 The mean score of communication service requirements and the HIS evaluation results
Full size table
Table 3 The mean score of system architecture requirements and the HIS evaluation results
Full size table
Table 4 The mean score of security service requirements and the HIS evaluation results
Full size table
Table 5 The mean score of system response time requirements and the HIS evaluation results
Full size table

Table 2 presents communication service domain as 10 items; showing a mean score of 3.49 ± 0.38 using the Delphi method and suggesting that the study HISs met 63.8% of the requirements of communication service domain.

According to Table 3, the 27-item system architecture domain with a mean score of 3.53 ± 0.37 was approved, which has been met in 65.5% of implemented HISs.

According to Table 4, the 31-item service security domain with a mean score of 3.5 ± 0.39 was approved, which observed in 72.4% of the implemented HISs.

According to Table 5, the 5-item system response time domain with a mean score of 3.82 ± 0.29 was confirmed, which observed in 76.2% of the implemented HISs.

Discussion

Experts approved HIS technical requirements in four domains, i.e. communication service (10 items), system architecture (27 items), security service (31 items) and system response time (5 items). Meeting the requirements in the study hospitals was found to be at good levels (68.9%).

Technical requirements for the HIS are largely addressed in literature using a researcher-made checklist, including a very limited number of items (4–7 items), on minor technical requirements [14, 17]. Even the requirements defined by the Iranian Ministry of Health and Medical Education in two domains of security services and communication services and 23 items [21] are not comparable with the number of items and domains defined in the present study. The present study designed data collection tools in a way that more specific and comprehensive domains and items were provided for evaluating the HIS on a national scale; nevertheless, technological advances require that further studies be conducted on diverse requirements and domains in the future.

The experts confirmed communication service as a domain of the technical requirements. The requirements for the study HISs associated with communication service were met in 63.8% of the cases, which was considered good though lower than the compliance rates of the other domains. Moreover, the standard protocol approved by the national healthcare authorities for patient information exchange was met in only half of the information systems of the research population. The National Coordination Office of Health Information Technology of America emphasizes the need for employing and observing standards and credible settings for interoperability and sharing and using electronic health information given the fact that these standards can prove useful in designing modern systems [26, 27]. Investigating the implementation challenges of the HIS found the poor integration of different systems to be a potential source of problems [28]. In Hungary, Aggod-Feko found HISs capable of communicating within hospitals, although they reported limited potential for inter-communication with outside the hospital [29]. Investigating the acceptability of the health information technology in seven industrial countries suggested that, despite their popularity, EHRs were yet lagging behind the health information exchange [23]. Numerous studies also reported very low levels of interoperability among HISs as low scores in assessments [30,31,32,33]. The lack of interoperability among HISs increases paperwork [34] and restricts healthcare organizations’ use of the product portfolio of a software supplier in a way that their data can be stored in the specific format of the supplier and cannot be easily transferred to other systems [35]. According to users, working with integrated systems could reduce working hours and increase work speed [6]. A high-performance information exchange system can indirectly contribute to nursing care [36] and reduce patient re-admission rates [37]. The HIS failure in terms of information exchange imposes financial burdens on hospitals and causes failure to refund the costs to healthcare organizations [38]. Exchanging information among HISs also helps reduce the costs of clinical communication among healthcare providers and transmission of laboratory and imaging reports [39]. Investigating Electronic Medical Records (EMRs) by Walker et al. showed that interoperability and health information exchange can result in an annual saving of $77 million [40]. A unified and centralized EHR approach for individual patients requires the provision of regional Electronic Patient Records (EPRs) with some degree of internal exchange [41], although the integration of HISs into other systems is known as the main feature of the fourth and current HIS generation [6]. A special attention should be paid to proposing a framework of communication service requirements for the HIS that includes information exchange standards at the system design phase given the following points: a) the key role of information exchange in information systems, b) the need for developing interoperable systems, c) prevention of organizational dependence on the products of a particular company, d) the effect of a lack of data integrity on treatment and costs and e) inhibiting the development and implementation of EHRs by heterogeneous software forests.

The system architecture was an important domain of the technical requirements. Compliance with the system architecture requirements in the study HISs were found to be proper (65.5%), which were yet lower than that of two other domains. In software development, system architecture determines the system development model and tools and environment development [42]. Multiple distributed systems developed in different programming languages demand more mechanisms in terms of practical communication [43]. Moreover, service-oriented architectures can be used to define IT infrastructures that enable different programs to exchange data and cooperate in the business process regardless of the type of the operating system and programming language through which the program is designed. Moreover, diverse healthcare programs can be run on the web based on messaging standards to exchange office and clinical data [44]. Systems designed based on diverse types of platforms and programming languages appear to make it difficult to develop HIS projects by creating numerous heterogeneous systems in an organization. Employing open-source tools, web-based implementation and supported standards as part of the framework of system architecture requirements can therefore assist with future software development.

Evaluating the information systems of the study population showed good levels (72.4%) of meeting security service as a domain of the technical requirements. Security issues hinder the application and implementation of information systems in the healthcare sector [45]. Security concerns associated with EHRs have also been reflected in literature [46,47,48]. Moreover, levels of technical security were reported as high in some health centers and moderate in certain hospitals [49]. In yet other studies, over 50% of respondents reported the security of their health data at good/moderate levels [32, 50]. Research suggests neither a specific policy for patient access to information nor a punishment for unauthorized access to information. Users can access the system using the same password with no time limitation given that no expiration dates are usually defined for the passwords [49]. The system must incorporate security mechanisms for controlling user access in terms of tasks or classified access [51]. An effective access control system ensures that only authorized users can access system resources [4]. On the other hand, the HIS should provide easy access to healthcare information at healthcare institutions [52]. The user-friendly techniques commonly used to guarantee data confidentiality and user validation include setting passwords, biometric techniques, smart cards and certifications [53]. The access control mechanisms attempt to prevent illegal operations before they occur [54]. According to the majority of IT experts, making a backup should be given top priority over the other security requirements of nursing data in EHRs [55]. Providing encrypted incremental backups for information was reported as a security measure in EHRs protection [56]. Given the effects of the HIS used for the security management of patient data on the care quality, the rights of patients and healthcare professionals and their working practices [57], patients may withdraw their information owing to their distrust of the HIS security, which is health-threatening [58]. Solving the security and privacy problems of the HIS is therefore crucial for meeting potential data protection challenges [59]. Hospital authorities should follow a standardized set of instructions to enhance information security [60, 61]. Different researchers have reported different security levels. Given the key role of data in providing patient care, data backups and data access control are known as the major security concerns of the systems. The implemented HIS should therefore incorporate a security framework that guarantees system security and facilitates access to information for authorized individuals.

The HISs evaluation showed good levels (76.3%) of meeting system response time as a domain of HIS ​​technical requirements approved by the experts. System response time is a major technical challenge of the HIS [28, 62]. The dissatisfaction with this domain highlighted in numerous studies was reported as an obstacle to applying EHRs and the main reason for their abandonment by users [62,63,64]. Organizations that heavily depend on computerized systems to deliver patient care require downtime as low as 0% and business continuity procedures to ensure safety and patient care continuity [65]. Timely high-speed system access identified as an important efficiency factor is therefore a general technical dimension of the HIS [51, 52]. A low system response time reduces the likelihood of the acceptance of the system by users and its successful implementation [66]. To enhance the system efficiency and consequently user satisfaction, the importance of the requirements for system response time is recommended to be emphasized when selecting the system.

Conclusion

A 73-item framework was designed to address all the HIS technical requirements in four domains, i.e. communication service, system architecture, security service and system response time. The possibility of defining and adding other items to this framework to meet the requirements for individual HISs makes it applicable to different settings. It can therefore benefit other countries depending on their degree of development and progress in implementing the HIS. These requirements can be used in the lifecycle of information systems to help customers select a system that meets minimum user requirements for the HIS. Determining these requirements can also assist the designers and developers of HISs in adapting the software program to user needs. Evaluating the HIS in the study population suggested good scores in meeting HIS technical requirements, although communication service and system architecture respectively received the lowest scores. Constantly developing heterogeneous HISs in hospitals makes the integration of these systems difficult to achieve given their different designs and their implementation according to different principles. The difficult transfer of the patient data stored in these systems to other systems is also a major challenge and obstacle to establishing interoperability among different systems in terms of developing EHRs and modifying the systems according to the individual needs of organizations and users. Sending the results of evaluating the HIS as feedback to software providers can assist them in improving their programs and meeting technical requirements.