1 Introduction

To provide patients with information, a wide variety of online health information tools, such as websites, patient portals, and mobile phone applications have been developed. As common diseases such as cancer, diabetes and hypertension are often diseases of older people [1], they are also increasingly confronted with online health information tools. These tools therefore play an increasing role in the everyday life of older patients. To illustrate, research dating back to 2002 showed that only 22 % of Europeans that were over 50 years of age had access to the Internet and only 38 % had interest in retrieving health information online [2]. More than a decade later, 85 % of the Dutch population between 65 and 75 years of age has access to the Internet in 2013. Of this group, 57 % uses the Internet to search for health information [3]. These numbers indicate that the medical digital divide is narrowing down.

In a recent systematic review we found evidence for the effectiveness of online health information tools for older patients (≥ 65 years) on self-efficacy as well as clinical outcomes (i.e., blood pressure, hemoglobin levels, and cholesterol levels) in four online health information tools [4]. These four online health information tools have in common that they consist out of multiple functions. Online health information tools for patients can have several functions, such as information provision, enhancing information exchange, and promoting self-management [5]. Online health information tools that were able to improve self-efficacy, clinical outcomes, or both, had in common that they had a ‘promoting self-management’ function, and additionally had a ‘providing information’ and/or an ‘enhancing information exchange’ function.

Now that we know that these tools can lead to positive outcomes in older patients, it is important to investigate how we can develop evidence-based online health information tools for this age group. Although we found that four online health information tools were effective, it is still unclear how the interventions were developed and how they were used by the participants. By distinguishing the useful, useable and used components of the interventions we can build a base for the systematical development of evidence-based online health information tools for older patients.

The first step in this process is to take a closer look at the development of the online health information tools that have been proven to be effective. More specifically, an important step that needs to be considered in the development of online health information tools is its usability as its benefits can only be realized if older adults can use them. Usability is an important issue to consider for this age group in particular, as older individuals have more problems using computer technologies [6]. The aim of this paper is therefore to give a more qualitative and in-depth overview of the effective online health information tools by evaluating the development process and the usability of these tools.

1.1 The Development of Online Health Information Tools

We evaluated the development of the online health information tools using the Medical Research Council’s (MRC) framework and the Spiral Technology Action Research (STAR). The Medical Research Council’s framework is a framework for the development of health-related interventions in general [7]. This framework distinguishes four key elements of intervention development.

  • The first element considers the development of the intervention by identifying existing evidence, identifying and developing theory, and modelling the process and outcomes;

  • The second element relates to the assessment of the feasibility of the intervention by examining the key uncertainties that have been identified during the development;

  • The third element exists of the implementation of the intervention;

  • The fourth element considers the evaluation of the effectiveness of the intervention.

Additionally, there are frameworks specifically designed for the development of web-based interventions. One example is the Spiral Technology Action Research (STAR) model [8]. This model describes the steps that need to be taken during the development of web-based health education and behavior change promotion. The model consists of five cycles.

  • The first cycle considers listening to the intended users of the intervention. For example, by understanding how users interact with existing systems;

  • The second cycle concerns the development of a plan for addressing the users’ needs and to identify the technical and organizational requirements of the intervention;

  • During the third cycle, the online health information tool will be developed. These three cycles relate to the first element of the MRC framework. At the end of this cycle, the first prototype will be developed;

  • During the fourth cycle, the prototype will be evaluated. This cycle can be compared with the second element of the MRC framework, in which the feasibility of the intervention is tested;

  • In the fifth and last cycle, the online health information tool will be launched and implemented. This cycle relates to the third element of the MRC framework.

These frameworks have in common that developing interventions is a holistic cyclical process. Van Gemert-Pijnen et al. [9] also argue for a holistic framework which takes the complexity of healthcare and the involvement of a wide variety of stakeholders into account. Also, both models have in common that interventions need to be theory-based. Using these frameworks, we will evaluate the development of the online health information tools that have proven to be effective in our systematic review.

In this study we will describe the development, the evaluation of the feasibility, and the implementation of the online health information tools, which corresponds to the first three elements of the MRC framework and the first four cycles of the STAR model. The last element of the MRC framework (the evaluation of the effectiveness of the online health information tools) is reported in our systematic review [4].

1.2 The Usability of Online Health Information Tools

As is mentioned before, in the development of online health information tools, their usability is an important issue to consider. Online health information tools that are easy to use for younger individuals might not be easy to use for older individuals, for instance because older individuals are less experienced with new media. To evaluate the usability of the online health information tools for older individuals we will use the guidelines that Pernice and Nielsen [10] have proposed to develop easy to use websites for this age group. They identify seven usability categories, of which four are specifically relevant for the development of online health information tools: (1) presenting information and text, (2) presenting navigational elements and links, (3) search, (4) and web address and home page. The other three categories are not relevant for the evaluation of the usability of online health information tools as they relate to webshops and the operating system or browser of the user.

2 Method

In our systematic review [4], we assessed the methodological quality of studies evaluating the effectiveness of online health information tools. We rated the studies that used a Randomized Controlled Clinical Trial (RCT) design as high or low quality. Next we performed a ‘Best Evidence Synthesis’ by attributing levels of evidence to the outcomes of the online health information tools (i.e., evidence, limited evidence, indicative findings, no/insufficient evidence). The level of evidence was attributed to outcomes of online health information tools for which significant improvements were found in two or more high quality RCTs. For four online health information tools we found evidence for the outcomes self-efficacy, blood pressure, hemoglobin levels, and/or cholesterol levels. To describe the development of the online health information tools we used articles that reported on the effectiveness of the four online health information tools from our systematic review. From the articles, we extracted the information about the content and development of the online health information tool. Next, we searched the literature for articles in which the development of the online health information tools was reported. To evaluate the usability of the online health information tools, we tried to obtain the online health information tools online. We created a codebook using the above mentioned four website design guidelines that were proposed by Pernice and Nielsen [10], see Table 1. However, the online health information tools were not publicly available (anymore) and no screenshots were reported that allowed us to evaluate the usability. We sent emails to the corresponding authors of the articles of the evaluation studies. Only one author responded and pointed out that their online health information tool no longer existed and had closed at the end of the evaluation study. Hence, we were not able to evaluate the usability of the online health information tools. Therefore, we will only report the evaluation results of the development of the four online health information tools for older patients which we found to be effective in our review.

Table 1. Codebook te evaluate usability of online health information tools for older patients
Full size table

3 Results

3.1 Description of the Online Health Information Tools

Of the four online health information tools, two were developed for patients with diabetes and two for patients with hypertension. One of the online health information tools for diabetes patients concerned a website where patients could enter their blood glucose readings, exercise programs, weight changes, blood pressure, and medication data [11, 12]. In case of changes, nurses could contact their patients via e-mail or instant-messaging/chat. The website also offered weekly online educational group discussion via MSN Messenger software. This online health information tool significantly improved self-efficacy, blood pressure, hemoglobin levels and cholesterol levels. The second intervention for diabetes patients concerned an online health information tool consisting of four functions: (1) videoconferencing with nurse case managers, (2) remote monitoring of glucose and blood pressure, (3) a web-portal providing access to patients’ own clinical data and secure web-based messaging with nurse case managers, and (4) access to an educational web site [13–17]. The use of this online health information tool also significantly improved self-efficacy, blood pressure, hemoglobin levels and cholesterol levels. One of the online health information tools for patients with hypertension concerned a home monitoring system where patients could send their self-measured blood pressure readings for review by their attending nurse or doctor, who in turn had the possibility to give automated patient decision support by text or email. Patients had the possibility to view their blood pressure readings on a website [18]. Patients that used this online health information tool had significantly improved blood pressure. The other online health information tool for patients with hypertension concerned a personal education program on a wireless tablet computer [19]. Patients received immediate individually tailored feedback on their medication use. Also, corrective strategies were printed and sent to nurses prior to the patient’s primary care visit. The use of this online health information tool significantly improved self-efficacy.

The development of these online health information tools will be described in the following paragraphs following the first three elements of the MRC framework and the first three development cycles of the STAR model.

3.2 Phase 1: Identifying Existing Evidence

All authors report that the online health information tools had been developed using existing empirical findings on how online health information tools can improve health- related outcomes. However, as there is little evidence, the empirical evidence that is used is very general or of low quality [18], making it difficult to predict whether the results will maintain for the specific older patient groups.

Moreover, a strong theoretical basis of the online health information tool has not been reported, with the exception of Neafsey et al. [19]. Based on the Social Cognitive Theory, they expected that their online health information tool could ‘enhance self-efficacy in patients to motivate them to adopt safe self-medication practices and modify adverse self-medication behaviors’ [19, p. 161]. The authors argue that animations form mental pictures in the patients mind and give meaning to their own self-medication experiences and will therefore guide their future self-medication behavior. In addition, the animations in the intervention have related multiple choice questions, which allow observational learning. Tailored interactive questions with feedback about self-efficacy will help patients gain confidence in self-medication. In line with the principle of ‘reciprocal determinism’, the learning is expected to continue during the visit with the nurse.

3.3 Phase 2: Assessing the Feasibility of the Online Health Information Tools

To the best of our knowledge, the feasibility of three of the four online health information tools has been reported. Bond [20] discusses the lessons learned from the development and the implementation of the online health information tool. During its development, the online health information tool was tested on its usability by focus groups, heuristic evaluations and think aloud sessions. First, a focus group session with 22 older patients with diabetes was held. They gave suggestions about the lay-out, the content and the design of the website. The prototype of the website was adapted according to participants’ suggestions. However, the author did not report which suggestions from the participants were retrieved and which elements of the prototype were adapted. Next, during the heuristic evaluation, experts identified usability problems using criteria and guidelines based on the W3C’s Web Accessibility guidelines [see 21]. Bond reports some examples of the feedback from the experts, such as the provision of a ‘contact us’ link and/or FAQ, prominent ‘how to use the site’ information, a ‘site tour’, a ‘forgot your password mechanism’, and a second navigation bar at the bottom of each page. Finally, the usability of the online health information tool was tested using think aloud sessions with five older participants that were recruited from assisted living facilities. Although the author reports that several problems were indicated by the participants, she does not mention which specific problems were mentioned and targeted. Moreover, the author does not mention the age of the participants or whether the five participants that participated in the think aloud session were patients with diabetes.

Starren et al. [22] describe the development objectives of one online health information tool. The first design objective relates to the usability of the technology. However, we found no literature concerning any usability tests with (potential) users of the system. The authors only mentioned one measure that was taken to make the system easy to use, namely a customized mousepad with four buttons that allows patient to answer video calls, to access the Internet, and to submit glucose and blood pressure.

Lin, Neafsey, and Strickler [23] reported on the usability testing of the PEP-NG eHealth intervention among older (≥ 60 years) patients with hypertension. The PEP-NG intervention was developed in four stages. In the first stage, the usability of the first pilot version of the prototype was assessed by five focus group participants. The second version of the prototype was adapted according to the results of the focus group. The results of the focus group interviews were, however, not reported. During the second stage, the second pilot version of the prototype was again tested using a focus group with new participants. Also, two participants were involved in a think aloud session. Using the results of the focus group and the think aloud observations, the first version of the prototype was developed for the formal usability testing. In stage three, this version was tested using a think aloud protocol and two focus groups. The second version of the prototype was adapted on the basis of the first formal usability tests. During the fourth stage, the second version of the prototype with a new sample of ten participants in a think aloud session. On the basis of these results, the beta-version of the PEP-NG was built. However, the authors have not reported the outcomes of the usability tests that they have performed.

3.4 Phase 3: The Implementation and Usage of the Online Health Information Tools

To our knowledge, the implementation of two of the four online health information tools was reported. For the implementation of one online health information tool [20], computers were installed in the homes of the study participants. The researchers faced some problems with the installation of the computers, such as viruses, a failing modem, memory board or printer, which was caused by a lack of software testing and damages to the computer during transportation. Next, the study participants received a training manual and one-on-one training to learn how to use the online health information tool. The training was based on computer classes for older adults. After the online health information tool was implemented, its performance was tested during the evaluation study. At that time some problems occurred. First, the website of the online health information tool had been hacked. However, the author did not report how this problem was solved. Second, there were problems with the hosting service. Participants received error messages and were not always able to submit their data. To resolve this issue, participants could sent the error messages to technical support personnel that resolved the problems and improved the online health information tool. The study lost one subject that was not able to enter data. For another online health information tool, the technical implementation has been described in detail [see 22]. The authors make the recommendation that the human component must not be eliminated during the implementation of the online health information tool. Adequate interaction between evaluation staff, implementation staff, and the telecommunication vendor is very important.

Some data on the usage have been reported, to our knowledge, for three out of the four online health information tools. For one online health information tool it was reported that the use varied widely under study participants [12]. However, the authors did not report if this influenced the outcomes of the study. With regard to the use of another online health information tool, the authors report a high compliance with the blood monitoring system, where participants submitted their blood pressure readings. However, the authors did not report whether the participants used the decision support information they received or whether they looked up their blood pressure readings on the website [18]. One online health information tool was used at the office of the healthcare provider. Participants were assisted in using the online health information tools where needed. The online health information tool was used before every visit to the nurse [19]. It is, however, not clear whether patients could use the online health information tool at home.

4 Conclusion and Discussion

The aim of this study was to evaluate the development and the usability of online health information tools for older patients, that have been proven to be effective. Following the MRC framework and the STAR model, the first step in intervention development is to identify existing evidence to create a theoretical and empirical basis. Most online health information tools were based on a scarce amount of literature and only for one online health information tool [19] the theoretical basis was reported. The next step in the development of online health information tools is testing its feasibility and more specifically its usability. Only usability tests of two online health information tools were reported. However, only the methods and not the results of these tests were reported. Hence, we were not able to identify the specific usability issues of the online health information tools. This information would be very useful in the development of new online health information tools for older patients. For two online health information tools, the implementation was described. Lessons learned from the implementation of existing online health information tools provide us with useful information for the implementation of new online health information tools for older patients. With regard to the usage of the online health information tools, the authors did not report any information about the use of the separate components. As a result we cannot be sure what the active ingredients of the interventions were.

It is important to consider that the evaluation of online health information tools is part of the development cycle and reporting about the evaluation is as important as the other parts of the development cycle. Therefore, we stress that authors adhere to existing guidelines to report on the development and usability of online health information tools in detail. Specifically, we recommend using the CONSORT EHEALTH guidlines [24]. According to these guidelines, eleven points are highly recommended or are essential to report on the development and content of the online health information tools. Authors should report (1) the names, credentials, and affiliations of the developers, sponsors and owners, (2) the development process (i.e., usability testing), (3) revisions and updating, (4) the source code and/or screenshots/screen-capture video, and/or flowcharts of the algorithms used to ensure replicability, (5) the URL of the application and/or details of where the intervention is archived, (6) how participants accessed the online health information tool, (7) the mode of delivery, features/functionalities/components of the intervention and comparator, and the theoretical framework, (8) use parameters, (9) the level of human involvement, (10) any prompts/reminders (e.g., letters, emails, phone calls, SMS) to use the online health information tool, (11) any co-interventions. According to these guidelines, the four interventions were described too succinct.

Regarding the usability, we were not able to evaluate the online health information tools, as they were not publicly available. Not reporting the online health information tool itself by reporting the URL, the place where the intervention is archived, or screenshots of the interventions, poses a threat for the replicability, which is essential in scientific reporting. Moreover, replication and synthesizesation of the results is important as this can help in the development of new evidence-based interventions. Because the development of online health information tools can be a complex process and a lot of different choices have to be made, we emphasize the importance of reporting the content of the online health information tool as detailed as possible.

The fact that we found that online health information tools were not available anymore also has an important implication for practice. Online health information tools are often developed with the help of (large) grants. When these tools have been proven to be effective in improving health-related outcomes, it is important that more patients can benefit from these tools. We therefore suggest that researchers put more effort in the dissemination and long term implementation of effective online health information tools for older patients.

To conclude, evidence exists that online health information tools can be effective in improving self-efficacy and clinical outcomes in older patients. Researchers are increasingly evaluating the outcomes of online health information tools [25], even for older patients [4]. However, it is still difficult to replicate the studies and synthesize results, as the online health information tools are not reported in detail. We therefore recommend to systematically develop online health information tools, which have a strong theoretical basis and which have been extensively tested on usability. Also, we argue that it is essential to report the content of the online health information tool in detail in a way that other researchers should be able to replicate the study. The CONSORT-EHEALTH is a useful tool to follow when reporting studies on online health information tools. Following these guidelines allows for the replicability of studies and the synthesizing of research results, and consequently a strong evidence base for the development of online health information tools for older patients.