1 Introduction

Digitalizing health care systems is considered to be a major means for meeting current challenges in health care [41]. Overall, the potential benefits of digitizing health care include increased access to care and the improvement in service quality. These are essential requirements for making health systems responsive and sustainable. In addition, digitizing health care systems has the potential for enabling the transition from treatment to prevention. In this paper, we present results from the ongoing research project INTROMAT (Introducing Mental Health through Adaptive Technology). Part of the project goals is to develop Internet-delivered psychological prevention and treatment programmes to people with mental health challenges or problems. One of the cases in the project is about increasing adaptive ICT support for eMeistring, a routine care clinic that provides guided Internet-facilitated cognitive behavioural therapy (iCBT) in secondary care for adults with anxiety and depression. INTROMAT is an integrated research project, with goals for implementing findings and new solutions for use in practice, and collaboration between academia and industry, to support in health care sector. The design ideas and solutions presented in this paper will in turn be developed further by an industry partner and implemented in the clinic and represents an example of a research and innovation loop, including clinical and user needs exploration, prototyping and user testing, and implementation.

Each therapist in the eMeistring clinic is responsible for providing support and treatment to approximately 15 patients. As one of the benefits of Internet-facilitated treatments is increased therapist capacity (about three times more patients per therapist), the therapists are in need of user-friendly and effective IT support. The clinical management system currently in use is mainly text- and list-based, and the clinic is in need of dashboards for improving the conditions for online clinical practice for both therapists and patients. This can include better overview over patient activity in the system, easier access to patients symptom development, and other indicators of patients who are in need of more support. These goals are also relevant to other health care practices; for example, health care professionals often do not get enough time to look into patients’ historical information. In order to improve the quality of service, health care professionals may be equipped with patient data analytics, including data visualizations. In this paper, we present the results from a study of clinical needs from health care professionals involved in guided iCBT and present a list of reusable visual components. We have conducted interviews with the therapists working in the eMeistring treatment programme to gather the requirements for a data support for therapists and also built insight into patient needs. We focus on the usability and reusability issues of supporting clinical mental health practice within guided iCBT.

Our aim is to support clinical practice in guided iCBT by providing data visualisations to therapists showing patients’ activities. Activities in this context mostly refer to what patients and therapists are engaged in while using digital treatment support systems. The underlying idea is that traces of digital activity and system-generated data can be used to raise awareness about important aspects for the clinical outcomes of mental health therapy. Again, these traces can be aggregated in the form of visualisations. Although there is a scarcity of this kind of work in guided iCBT, examples can be found in other fields. For instance, in educational research, the field of learning analytics focuses on data-driven ways of improving educational outcomes [35] by collecting and analysing traces of what learners leave behind in digital systems [34], and finding ways of making the data visible and useful. Charleer and colleagues [9] have studied learner dashboards for students and found that visualising student effort (i.e. produced materials, time spent, etc.) is only helpful when it highlights how the effort contributes toward the intended learning outcomes of what is being studied. They furthermore find that solutions that empower students and increase their ability to reflect and make decisions have a more positive effect on motivation, than, for example, automating a learning trajectory based on data. Corrin [11] and colleagues have studied how analytics can be integrated with a teacher’s learning design and argue the necessity of matching the data visualisations with the pedagogical intent of the teacher. CBT and education share the notion that one of the major change processes or facilitators of improvement is human learning.

1.1 Usability

In computer science, a common approach to assess the value of an application is to evaluate its usability. Poor usability and lack of user-centred design have been described as two of the reasons for low engagement with mHealth apps [38]. In general, ICT with poor usability can lead to situations of low goal achievement efficiency or the application not being used or being rejected. Usability studies are grounded not only in the social and behavioural sciences but also in the science of design [27]. Through the approach of research-through design, it is possible to explore ideas to improve practices by building artefacts to support the practice at the same time as ensuring their relevance and validity [43]. This can be ensured by engaging with practitioners within the addressed field, in design of the digital artefacts. A recent review of usability practices in design of digital health applications [19] found that end users such as patients seldom are involved in the design of applications, although they are often involved in post-development evaluation. Here, we advance the state-of-the-art mobile health (mHealth) development practice by engaging therapists in the design of the digital environments that are being used to mediate guided iCBT.

1.2 Reusability

Model-based approach may play a significant role in supporting digital health. In current practice, data analysts need to spend a vast amount of time processing data for analysis and producing effective reports. In this paper, we present a model-based approach to develop reusable visual components. With this approach, a data analyst will be able to incorporate visualizations for representing results throughout the process of data analysis. This technique allows the user to visualise data from various levels of abstraction. For instance, it allows grouping of activities based on an ontological hierarchy, which permits data visualization from a higher level of abstraction. The visual components are equipped with temporal sliders, which allows a user to intuitively follow how patient data evolve over time. Since our work is related to the topics of visual analytics [22], we underscore that our focus for this paper is in the overlapping part of three research areas which include visual analytics and usability, digital health and model-based information system development (see the Venn diagram in Fig. 1).

Fig. 1
figure 1

The focus of this research

Full size image

2 Methods

We have overall carried out the research inspired by methods from design science research (DSR) [17]. DSR aims to combine behavioural research with design research, in order to “create innovative artefacts that extend human and social capabilities, and aim to achieve desired outcomes” (ibid., p. 1). As such, it articulates a process to carry out research-based technology innovation. DSR is centred on the development of novel artefacts supported by the rigorous research in the knowledge space and also relevant for the society. Zimmerman and colleagues [43] argue that one criterion for evaluating the contribution of a DSR project is assessing the relevance of the developed technology, and where design knowledge resides in the artefact [14]—something that will transform the world from the current state to a desired state. Hence, we have taken an iterative, longitudinal approach to study the practice of guided iCBT at a mental health clinic, in order to understand how to best support the clinical and therapeutic practices with visualisations of relevant aspects of their clinical work. Additionally, we have been interested in how to improve the online environment for the patients. We have furthermore engaged in collaboration with practitioners at the clinic in drafting and evolving tools that can help them gain insight into their patients and make improved clinical decisions. By engaging online therapists in the process, we aim to increase the utility of the solutions.

2.1 The development process

The overall process, modelled in Fig. 2, has taken place in three stages. First, we arranged a series of collaborative workshops together with representatives from the clinic to develop a joint understanding of the need for clinical decision support, and what would be helpful to patients. Two to three clinical experts took part in each workshop, in addition to the researchers. This stage produced workshop notes containing descriptions and ideas about clinical needs. In the second stage, we created a set of what we label first-generation visualisations, which were based on the findings in the workshops, and our understanding of the types of data that their content management system produces. These were then presented to the clinicians, who provided feedback that formed the basis for further refinement of the first-generation designs. The second stage included several iterations of refinement and feedback. We consider the draft visualisations and feedback notes as the main body of data in this stage. In the third and so far final stage, we created a web page resembling the existing clinic management system with clickable versions of the first-generation visualisations. The web page was populated with mock, but realistic, data. Then, we arranged an evaluation of the visualisations in the form of a collaborative usability inspection with the clinical psychologists at the clinic.

Fig. 2
figure 2

Application of design science research approach

Full size image

The evaluation process followed methodological inputs from group-based expert walkthrough, as described by [15]. This scenario-based usability inspection methodology walkthrough is related to cognitive walkthrough [40] and pluralistic walkthrough [6]. The goal is to capture the expert knowledge of a group of evaluators to identify potential usability problems and, importantly to our work, design improvements and successful design proposals. The domain experts’ contextual knowledge is a key input to the evaluation and as such is a methodology well suited to early evaluations of applications directed at particular work domains. Input from domain experts may inform designers not only on usability issues, but also on the perceived utility of the ICT being developed. The evaluation is structured around task scenarios where usability experts lead the domain experts through the object of the tool, and normal use context.

The walkthroughs were conducted using a videoconferencing tool (Zoom), using a semi-structured interview guide, and were recorded using the same tool. Six participants took part in the expert walkthrough and were recruited by one of the clinical therapists who took part in the first two stages of the design research process. Two evaluators and one domain expert participated in each interview.

2.2 The structure of the evaluation process

Two to three days before the interview, the participants were sent a document called ’Interview guide’. The interview guide contained information about the purpose of the evaluation and explanations of the actual prototypes. The participants were asked to read the interview guide before the interview. During the interview, we first informed about the purpose of the prototypes. Then, the domain experts were informed about the purpose of the evaluation which includes the evaluation of the intuitiveness and learnability of the prototypes and feedback on potential areas of improvement.

During the evaluation, each visualisation dashboard element was presented by the theme (described in Sect. 3.1). Interaction with the dashboard was done by the domain expert, whilst they shared their screen. This was done to increase the evaluators’ experience of detail and immersion with the digital environment. Whilst the domain expert clicked around in the dashboard prototype, a discussion of the dashboard element took place, and the evaluators used questions from the prepared interview guide to guide the discussion. Each interview was reviewed by the researchers immediately after, where the main observations were discussed, and critical updates to the visualizations were made.

2.3 Development of reusable visual software components

The outcome of the first-generation visual design artefacts was used for the study and development of second-generation visual software components, libraries and architecture. The development is being carried out considering multiple clinical treatment programmes in mind. Therapists and researchers involved in across several treatment programmes are involved for the development of artefacts in this phase. The INTROMAT project addresses five use cases:

  • Relapse prevention for bipolar disorder;

  • Cognitive training in attention deficit hyperactivity disorder (ADHD);

  • Job-focused treatment for depression in adults;

  • Early intervention and treatment for social anxiety disorder in adolescents; and

  • Psycho-social support for women recovering from gynaecological cancer.

In INTROMAT, the project participants developed Internet-based treatment modules for the above-mentioned cases. Several IT vendors were involved to support the randomised pilot studies across the five use cases. We argued for a reference architecture for carrying out the activities of Internet-based treatments. Suresh et al. presented a reference architecture which generalises the concepts of treatment modules and includes software components for the development and execution of Internet-based cognitive treatments [29, 30]. However, in practice it is assumed that several IT vendors would be involved in implementing the software systems for treatments. Therefore, there is a need for generalizing visual components which can be reused across different use cases. In this paper, we present the need for a model-based framework for visual components since we need to deal with the heterogeneity of data structures from several IT vendors. We propose to use model-based approach to accommodate generalised visual component libraries. We propose data models that cover the concepts related to iCBTs in general, and we present a conceptual framework to support the development of visual components and their interaction in between.

3 Guided iCBT

There is a growing trend to provide Internet-facilitated cognitive therapy programmes. The effectiveness of these services is expected to be affected by their usability and reusability. The usability approach is about ensuring that iCBT services are easy to use, effective, efficient and satisfactory. It is important for the success of Internet-facilitated interventions. It can help to improve the intervention in terms of preventing unnecessary errors and inconsistent design. Reusability, in this context, refers to the likelihood that existing visual components for supporting iCBT can be used again in some form with slight adjustments or no modification within the development process of other iCBT services. Usability and reusability issues are, therefore, critical to the development of effective and efficient iCBT services and to supporting clinical mental health practice within guided iCBT.

In the study presented here, we focus on just one of the cases: eMeistring, when developing visual components for supporting iCBT. However, we are involved in several other case studies being conducted in the mental health domain within the project scope. Focusing on usability and reusability, we show how to design usable visual components and adapt them, as reusable assets, to other relevant cases in the project. In this section, we present how we engaged with the therapists in the programmes.

3.1 A mental health clinic for guided iCBT: eMeistring

The eMeistring clinic offers guided online cognitive behavioural therapy for the mental health problems of panic and social anxiety and depression. The effects of the CBT on the patients’ mental health are considered positive and long-lasting, also when compared to face-to-face therapy, in line with findings from recent scholarly literature [1, 2, 18, 31, 37]. There are issues with patient dropout, however, also in line with scientific literature findings on mHealth and online mental health therapy worldwide; see e.g. [20, 32]. In the long run, our work is intended to contribute towards lowering the dropout rate, increasing the percentage of successful therapeutic outcomes, and enrich the opportunities for interaction between the patients and the therapists. In cases where the iCBT treatment is deemed to be non-optimal for a particular patient, which the clinic occasionally experiences, we hope to contribute a basis for a decision on deferral to other treatment more quickly. At the level of interaction between patient and therapist, we assume that there are particular conditions to counselling when those involved do not meet or interact face-to-face, but only interact through writing text in a web-based system. These activities can be scaffolded more or less ideally, and our approach is to exploit the opportunities offered in visualising data.

Patients are admitted to the clinic by their general practitioners or other clinical specialist referral. It is also currently possible for patients to contact the clinic directly without a referral. The treatment programme lasts 14 weeks and consists of eight modules covering aspects of mental health problems and CBT. The main activities that the patients are engaged in are reading and reflecting on their mental health problems; completing training assignments about the content of each module; and behavioural elements such as behavioural activation (BA). BA is a treatment strategy against depression that involves scheduling of activities that are important and pleasurable to the patient, to work against symptoms of withdrawal, explained in further detail below. Additionally, they complete self-assessment (currently MADRS) once a week. All activities except BA are mediated through a text-based clinical management system. The BA module is paper-based. Each patient is assigned a therapist, who assesses the patients’ progress and provides personal feedback via messages every week. The therapist additionally decides whether a module is to be considered as completed by a patient, and, if yes, subsequently assigns the next module.

As mentioned, the practice of online mental health therapy is based on different conditions than face-to-face therapy. For example, the interaction between the patient and the clinician in face-to-face therapy is very much temporally and spatially tied. There is a dedicated hour and place for the therapy, which encompasses the relationship between the clinician and patient. In guided iCBT, the patient–clinician and patient–therapy relations are in many ways sustained temporally and can take place anywhere. One of the treatment strategies in use in the clinic is behavioural activation, which is a common strategy used for treating depression. Behavioural activation [10] is a sometimes stand-alone component of CBT and involves the “scheduling of pleasurable activities to increase contact with sources of positive reinforcement” [21, p.361]. Ideally, the therapist should be aware of the correlation between the patients’ scheduled activities and symptoms, and in guided iCBT, this involves making use of the available data.

3.2 Designing clinical dashboards: a user-centred approach

Based on our workshop-based exploration of the problem space in collaboration with therapist representatives from the clinic, we arrived at three main ways of how patients and therapists can be supported with activity data visualisations, and a set of proposals of how to concretely visualise relevant information. The visualisations are drafted as snippets, which easily can be integrated with the digital system in use by the therapists. The following information need categories were identified:

Providing therapist insight into group of assigned patients The therapists will presumably be in a better position to support the patient therapeutic process the more they know about the patients needs, development and activities. This need can be exemplified by quotes such as “How do I choose the right person (to treat) first?”, and “The least active patients are the least visible in the system”. In the current version of the therapy management system, the traces of patient activity available to the therapists are messages sent between them, weekly self-assessment screening results, and patient diaries and responses to tests tied to each module (i.e. “what have you learnt in this module”). It is possible, however, to provide more detailed information, based on the data produced by patients and therapists while using the system. System needs exploration carried out with therapists for this project revealed three main types of therapists’ needs for insight into patients: 1) a way to prioritise who to help first of the patients; 2) to know about how each patient is progressing with the therapy; and 3) to know how much time and effort the therapist has spent on each patient during the therapy trajectory. The first need arises partly because the therapists do not have access to any kind of aggregated views of their patients in the system, and partly because the patients have individual needs for example for follow-up for the duration of the therapeutic process. The state of each patient must currently be assessed by reviewing direct responses to self-assessments and diaries, etc. The same observation is the cause of the second need for information. The clinic experiences a high dropout rate (around 60 per cent complete the therapy), a common phenomenon in iCBT [37], and has a stated goal of lowering this number. Currently, the therapists have access to the information provided above, in addition to whether the patient has completed a module or not. Insights into each patient activity will enable the therapist to intervene and assist with advice and encouragement, for example in cases where progress is not taking place as expected. The third and final category is insight into how much effort has been exerted by each patient. This is a way for therapists to learn about how much progress can be expected for their patients. It enables the therapist to self-reflect and adjust treatment strategies to ensure a constructive balance of efforts between each patient. Currently, the only source of feedback on this issue is personal memory.

Proposed visualisation Figure 3 represents a generated view of the progress and activity of each of the patients assigned to a therapist. It is intended to support making decisions about who of the patients to prioritise. The concentric circles each indicate one week of the programme (14 in total). Each segment or “cake” in the circle indicates a patient. The colour in each segment indicates how far the patient has gotten since starting. The colour (red–grey–green) and colour grading for each patient indicates trends in the MADRS score, red is negative, green positive and grey indicates stable values. Visualising trends in MADRS scores is based on the previous work of Grieg et al. [16] about supporting guided iCBT with visual analytics. The black lines indicate how many of the modules each patient has submitted. Comparing with the background colour tells the therapist whether a patient is on, ahead or behind schedule. The thickness of the black line indicates how much time the patient has spent online in the system. The grey shadow behind each black line indicates how much time the therapist has spent on each person.

Fig. 3
figure 3

Therapist overview of patients

Full size image

Although the visualisation has the advantage of presenting patient activity and progress data in a condensed way, there is a threshold to how many patients it can present at the same time. From a usability perspective we estimate that it will scale well up to 15 patients, before the information becomes too condensed. However, in cases of increased number of patients for the therapists, we proposed an alternative visualisation where the same information is presented in a tabular format with patients listed vertically, and progress and significant events are presented horizontally.

Fig. 4
figure 4

Alternative visualization for spider diagram

Full size image

The alternative visualisation demonstrates patient names including their weekly progress, latest time spent and total time spent on the iCBT programme, and total messages from therapist to patients and vice versa. The numbers in the table shows which module each patient is currently active in. Consistent with the spider diagram, the colour in each week indicates how far the patient has gotten since starting the therapy. The colour grading for each patient indicates trends in the MADRS score, red is negative, green positive and grey indicates stable values (see Fig. 4). The proposed alternative solution may help the therapists easily access an overall picture of patients’ performance during the therapy trajectory. It is intended to support the therapists who have a problem with following the spider diagram to use this tabular format to monitor view of the progress and activity of each of the patients assigned to themselves.

Supporting therapist insight into patient activity and development In addition, to having an overview of all patients to be able to prioritise between them, therapists also have a need for insight into the activity and development of each individual patient. Currently, the insights are based on the patients’ responses to the module tests, their patient diaries and the MADRS results. The patients additionally keep behavioural activation diaries, but this information is currently paper-based and outside the system they use. The idea is that by visualising the relevant information, the therapists will have better bases for making their therapeutic decisions, and additionally will have further opportunities to make interventions when patients are in danger of dropping out.

Proposed visualisations Figure 5 is a visualisation proposal that collects items from the patients behavioural activation diary, the categories of activities and patients MADRS scores. The purpose of this visualisation is to find connections between scheduled activities in the BA diary of a patient, and positive development of MADRS scores. This allows the therapists to click on a specific range of MADRS score and find the patients whose scores are in that particular range. The figure highlights patients with id Patient103, Patient113, Patient114 since their MADRS scores are in range 30-34. Therapists can subsequently click on any of these patient id’s to find out what are the activities or category of activities the patients were involved in. The purpose of this visualization is to enable therapists to quickly browse the data and extract insightful information related to activities and self-assessment scores. This visualization also enables therapists to see which activities work well and vice versa.

Fig. 5
figure 5

Connection between patient activity and MADRS score

Full size image

Additionally, the therapist have a need to see which of their patients are in danger of dropping out.

Figure 6 is a draft for a list containing the patients assigned to a therapist who are behind on their modules. This can, for example, be generated by listing the patients who are behind a specified threshold of expected modules completed, or by listing all patients who are behind with their modules. The list contains a link to the patient page of the persons in question, along with indication of how much time they have spent in the system (green bar), when the patient last logged in (red bar—a larger bar means that it has been some time since users logged in), last module they submitted and submission date.

Fig. 6
figure 6

Patient dropout warning list

Full size image

To provide the therapist insight into how a patient works during the week, we have drafted a table where days of the week are indicated by letters vertically on the left, and hours of the day are displayed horizontally at the bottom. The green bars indicate when patients are online and working in the iCBT management system. The green circle with a module number indicates that a module is completed. The email icons indicate when messages are sent (with a blue arrow) and read (with an orange arrow). This could be further augmented with data about the platform used when accessing the system, as there are different conditions to system use, for example, when using mobile platforms compared to a PC (Fig. 7). (The system can be accessed using any platform.)

Fig. 7
figure 7

Patients weekly activity

Full size image

Motivating patient persistence We have also aimed to increase the amount of visual feedback provided to each patient. The goal has been to increase the likelihood of successful therapeutic outcomes by increasing the support offered to each patient, particularly by visualizing their progress. The needs of the patients, as expressed by the therapists, can be exemplified as: “Am I doing too much, am I doing too little, am I on the right track?” and “What have I delivered, compared to what I am supposed to deliver?” The current source of feedback offered to the patient comes in the form of qualitative assessment messages from the therapists. We aimed to provide more day-to-day and direct feedback based on the activity levels and kinds of the patient and to increase the patient motivation to continue the therapy.

Proposed visualisations To support patients persisting engagement with the CBT, we propose a refined version of a relatively simple and well-known visualisation of progress—a progress bar. It compares actual progress with expected or planned progress, in addition to visualising the amount of messages to and from their therapist. This visualisation is also reported as interesting to therapists, to see if one particular patient is progressing as expected, in a simple way. The circles in the middle of the progress indicate total modules of therapy. The green circles with a module number indicate completed modules. The bigger circle with a dark red colour refers to active module information and grey circles are forthcoming modules (see Fig. 8).

Fig. 8
figure 8

Therapy progress bar for patients

Full size image

In the figure, vertical slider indicates generic progress as expected, measured by counting weeks from the start. The actual patient progress is indicated with the circles and measured by completed modules. The messages are indicated with differently coloured arrows, with the patient messages at the top and therapist messages at the bottom. By studying the week numbers at the bottom of the figure, we see that each module is tied to a week and that time beyond week 8 does not have modules tied to it. This second-generation visualisation is a refinement of a previous version that distributed modules equally over 8 weeks. During the collaborative expert walkthrough, we discovered that the therapists had an ideal that the programme was completed over 8 weeks, but allowed an extra 6 weeks to cover for unforeseen contingencies encountered by patients. The therapists’ observation was that the clinical effects for the patients were the best when they completed the programme by intense work over 8 weeks, and the revised progress bar is intended to reflect that.

3.3 Relevance to other treatment programmes, e.g. RestDep

Cognitive impairment is a core symptom of depression and may remain as residual symptoms, such as attention, memory and executive deficits, after the main symptom is treated. Experiencing cognitive residual symptoms could negatively affect daily functioning and be a risk factor for developing new episodes of depression. RestDep is an Internet-delivered cognitive enhancement programme for residual cognitive symptoms after major depression. The programme consists of 10 modules. The content of the programme is in the form of texts, video and exercises. All exercises given in each module aim to help the patient cope with the cognitive difficulties. The participants were encouraged to complete two modules each week and to finish the programme within 5 to 7 weeks.

Similar to eMeistring case, the main activities that the patients are engaged in the RestDep are reading and reflecting on their mental health problems, and completing assignments about the content of each assigned module. Additionally, self-assessment scales such as The Behavior Rating Inventory of Executive Function-Adult (BRIEF-A) which comprises the Behavior Regulation Index (BRI) and the Metacognition Index which could be combined into the Global Executive, and the Rumination Response Scale (RSS) are used to evaluate patient progress during the intervention period.

In order to increase patient adherence, therapists’ support for complex decision processes is one of the main needs that clinicians encounter when they treat patients for various kinds of problems. Decisions may include patient preferences, goals of the intervention, optimal duration of the treatment or psychometric tests, and effective visualization may help clinicians operationalise the adaptation of treatment in terms of these decision-making points.

Usable visualization assets designed in this exploratory study have, therefore, potential to supporting therapist insight into group of assigned patients and individual patient activity and development, as well as motivating patient persistence in the RestDep case. This may be generalised to other settings or context where therapists and patients have similar needs and requirements, we elicited in this study, during the therapy trajectory.

4 Expert evaluation outcomes

Before we conducted our evaluation, the first-generation visualisations were developed as an online, interactive version, laid on top of manipulated screenshots from the current version of the system in use at the clinic. Overlaying the visualisations on screenshots from the clinical tool in use was to make it possible for the evaluators to envisage how the prototypes could be integrated with the tool in use now. We used three screenshots from the existing clinical management tool: home page for therapists, home page for patients and patient view from therapists perspective. The starting point for the interactive version was the spider diagram. Clicking the coloured backgrounds in the spider diagram took the user to the patient progress view. Clicking the progress bar in the spider diagram took the evaluator to the weekly patient interaction view. Clicking the “Flags” menu item took the evaluator to the patient dropout alert list. Finally, the “home” icon took the evaluator to the spider diagram. During the evaluation, additional pages such as an alternative table view of the spider diagram and the new version of the patient progress bar were presented. Finally, we added “tool tips” activated on mouse-over of each feature, where a short explanation of the feature was provided.

After each interview, we reviewed the conversations and updated the visualisations. The updates were limited to “critical” features, or where we discovered that our original ideas were made on incorrect assumptions and would apply to all evaluators, or in one case where we provided options to cater for more individual preferences.

The results of the group-based expert walkthrough are presented here:

Increasing relevance of the patient dropout alert list One of the initial ideas to increase the therapist insight into the collective state of their assigned patients was to create a list of the patients in danger of becoming inactive and dropping out. The data source envisaged to capture this was “overdue modules”, indicated by red dots in the initial visualisation (see Fig. 9). This was changed to “time since last submitted module”, based on the aforementioned feedback on the therapy process that the patient was never assigned a new module before completing the previous and having it approved by the therapist. Additionally, the visualisation included how much time the patient had been online, presented by a bar and a clock value. We received feedback that it was more interesting to see when the patient last logged in, in a list such as this, and added this to the visualisation. For instance, a therapist stated that “The dates usually don’t tell me so much. I think it is hard to remember just the date today. So if it could be like it was submitted 10 days ago\(\ldots \) Or\(\ldots \) 14 days ago, or 3 days ago, that would help me more”.

Fig. 9
figure 9

Patient dropout warning list, before evaluation

Full size image

Improving design of the patients weekly activity table We aimed to provide direct feedback on the patients activity levels and to increase the patient motivation to continue the therapy. A therapist stated that “we would like to give the patient some visual information that users are on the right track, and to see how much they have completed. I think that’s gratifying for most users”. Similarly, another therapist stated that “I think I would use it for to see whether they have been active. That would save me quite a lot of time. I can just look at the overview and see okay the person hasn’t been active”.

Furthermore, we received feedback that it was useful to see module name completed in the week and to visualise messages between therapist and patients with colours. For instance, a therapist stated that “It’s enough information for us that they complete the module and whether they are active or not, rather than when they are active”. Figure 10 illustrates the appearance of the weekly progress table before the evaluation. The table illustrates how much time a patient spent on the therapy programme, sent and received messages. A blue dot represents completed module information in the week.

Fig. 10
figure 10

Patient weekly interaction, before evaluation

Full size image

Reduced complexity and increased relevance of the progress bar The patient progress bar for therapists was considered too complex for their needs. It was revised and made simpler without losing information. Concretely, we removed the arrows indicating weeks and also reduced the size of the e-mail icons, to reduce the amount of elements needing attention. Additionally, we received the feedback that it was irrelevant to present more than one overdue module, as there would never be more than one overdue module, because starting a new module was reliant on having completed the previous.

Aligning the progress bar with progress goals Figure 11 illustrates the appearance of the progress bar for patients before the evaluation. The progress bar initially displayed the eight modules of the programme equally divided over the 14 weeks. Initial response to this was that the time allotted for the therapy was indeed 14 weeks, but the patient often experienced better outcomes when completing one module a week, which was also the intention of the programme. A therapist stated that “Ideally we want them to complete one module each week. It’s really seldom that they do that, but our plan is to get like, if there is 8 modules, we want the treatment to be about 8 weeks. But there is always something that happens. So they might get sick, and lose their progression for some reason. And the 14 weeks is sort of their buffer. So they can actually drop off for a while, and come back in and still be able to complete it. So the interesting question would be if we sort of sell them the 14 weeks, they might sort of take more time, or, yes. Maybe there should be like a, at least from the patients view\(\ldots \) This (progress bar) is great for us. But for the patients view at least, it could be a dotted line of ideal finishing time”. Hence, we modified the patient progress bar to underscore the intention of completing one module a week, and indicating that the time following the 8 weeks was “extra time”.

Fig. 11
figure 11

Patient progress bar, patient view, before evaluation

Full size image

We updated most of the visualisations during the process of evaluation, the only figure that was not updated was the spider diagram. A therapist stated that “Okay, that makes sense. That is really like\(\ldots \) Then I can easily see that these four patients here they are not doing well, I should prioritise them”. Similarly, another therapist indicated that “Immediately I see which patient I need to address first, that’s the red ones, yes, and this one indicates that the patient hasn’t done much work either. So I will get that instantly”.

It would be beneficial with a way to see more clearly which patient was which in the diagram, but we decided that this could be redesigned at a later stage, or implemented directly in the final version. Summing up the evaluation points focused on in the interviews, the evaluators scored the prototypes high on potential value to their work, expressed that they looked forward to seeing them in the tool that they use for clinical therapy, and that it would be helpful to them in the clinical work. The intuitiveness and ease of understanding of the visualisations received more diverse feedback. There were individual preferences involved in how to understand visual presentations, some prefer text and tables, others prefer images, for example. Here, the future challenge is to cater for the preferences of all the clinicians, as much as possible.

5 Visual prototype development with model-based approach

We propose to use domain-specific models for dashboard components. Dashboard components, i.e. visualizations and data analysis techniques, are associated with an information model. In Fig. 12, we present the architecture of our system where we articulate the client–server communication by means of an application programming interface (API). A library of model-based visual components are available in the server. When a client, e.g. browser, requests for a visual artefact, the server sends the scripts for rendering graphics in the client device. Server application fetches relevant data from existing health care database and transforms them into appropriate model for visualization. The server maintains the status of the visual component running at the client device. The server application is featured with the following:

  • Support for abstraction by using ontologies and dimensional modelling [33];

  • Support for cause analysis using data mining techniques

Besides these features, the visual components are equipped with temporal sliders which enable the user of the system to see the progression of events for a particular time period.

Fig. 12
figure 12

Architecture design of model-based dashboard system

Full size image

The proposed architecture describes the design of our solution space. Figure 13 illustrates how model-based approach can be applied in various stages of implementing our system. The figure is adapted from [7] where the concept of extractor and injector was introduced. The idea of using an extractor is to represent the availability of appropriate software artefacts that are able to extract knowledge from a technical space and be able to inject such knowledge in another technical space (called injectors). The problem space consists of requirement specification and domain model. The requirement specifications have been articulated in previous two chapters by means of visual artefacts that need to be developed for visualizing health care information. To implement the visual artefacts, we used Data-Driven Document (d3.js) libraries and existing process mining tools (e.g. Fluxicon Disco [13]). We extracted domain models for visualization which are formally specified as directed graphs. Our domain models consist of ontological and/or dimensional information which allows the hierarchical organization of data. The domain models are transformed into suitable format (e.g. CSV, JSON code) for incorporating them into visualization software. We utilised both model-to-model (M2M) and model-to-text (M2T) transformation for converting health care information into suitable data format for the visual components. M2M transformations are used for transforming the domain model to an abstract representation by means of aggregating information based on ontological hierarchy and dimensional information. M2T transformations are used to transform a domain model from its graphical representation to a data set in JSON/CSV format which are compatible with the visualization software.

Fig. 13
figure 13

Technical spaces and coverage

Full size image

As mentioned above, in our approach visual components are associated with domain model; Fig. 14a presents a domain model for the proposed spider graph. We will refer to this visualization as Vis-Spider. Model transformation techniques can be used to extract this information from an existing health information system and instantiate this domain model. This visual component needs to be connected with other parts of the system, such that the system should allow selecting a patient from the cake view and see the details of patients completion of modules or the correlation of patients symptoms with self-assessment score. The API at the server side mediates the communication between a variety of visual components.

Fig. 14
figure 14

Domain-specific model for a Vis-Spider; b Vis-EFlow; c Vis-Corr

Full size image

Figure 14b presents the domain model for visualizing event flow. We will refer to this visualization as Vis-EFlow. The events are associated with case id (i.e. patients identification), time stamp (i.e. event time), activity and resource information. Many existing process mining tools use event logs that include these information [39]. In this domain model, we have incorporated dimensional information for activities. This allows our event logs to be organised hierarchically. The incorporation of dimensional modelling in event logs permits us to group activities and view information from a different perspective. The concept of dimensional modelling originated from data warehousing and business intelligence (DW/BI). Organizations embraced DW/BI techniques in order to handle large amount of information. Dimensional modelling allows us to incorporate the following features:

  • organization of large amount of data

  • process raw data in various ways and turn them into useful information

  • show correct information to the right person

  • provide useful knowledge to help decision making.

The DW/BI systems emphasise collecting and processing raw data quickly and turn them into useful information while preserving the consistency of the data [23]. It has been widely accepted by the BI community because of its simplicity and performance in presenting analytic data. In our approach, we propose to use dimensional modelling for organizing health care information, e.g. filtering and grouping events based on patients diagnosis, activities, etc. In our case, dimensional models package the data in a format that allows simplicity for displaying understandable information to users and also supports developing efficient data analytic tools in terms of query performance. Our event model allows us to change the level of abstraction in the event logs. We utilise this feature of dimensional modelling for specifying event flow analysis requirements. The purpose of this dimensional model is to provide an easy-to-use visualization for its user to investigate care flow from different context. We propose to use ontological hierarchies to provide hierarchical representation of health care information along each dimensional model. Traditionally, fact tables are used to store data at the lowest grain, e.g. records about physical activity or events. Fact tables always use foreign keys to associate the records/events with their dimensional models. Figure 15 shows a dimensional model where we incorporated health care ontologies, e.g. SNOMED-CT and ICD-10 ontologies. Fragment of the SNOMED-CT ontology is shown in the figure that links a data from a dimensional model.

Fig. 15
figure 15

A dimensional model for specifying data mining/visualization requirements in health care

Full size image

In Fig. 14c, we present a visualization called Vis-Corr to study correlation of patients’ activity and self-assessment score. Activities are recorded hourly by patients in a diary, as part of their behavioural activation. In eMeistring, this visualization can be used to see how activities carried out by the patient correlate with their MADRS scores (or symptoms), or in other words which activities play a role in reducing the symptoms of depression. Since eMeistring allows patients to write free text for activities, the number of nodes representing activities could be very many in the visualization. To deal with this situation, we propose to use an activity ontology [42] which will allow hierarchical representation of activities in the visualization. The visualization with a temporal slider allows therapists to investigate the effects of various activities and their correlation with depression symptoms. In future, we will incorporate a data mining technique which will extract patterns and visualise them with Vis-Corr. For example, the therapists would be able to see if activity-a and activity-b play a major role in the reduction in depression symptoms. Many iCBT treatments are based on the principle of behaviour activation. However, therapists currently do not have a visual tool support to investigate how their patients are practicing them. Our proposed method will allow therapists to investigate more into this.

6 Discussion and related work

We did not use any real patient data in our study, but the consequences of implementing the ideas we propose would necessitate consideration of ethics and safety risks to the patients. One aspect is tied to sharing of personal health information by patients. This needs to be based on the rules and regulations of the GDPR law as specified by the EU, especially that data use is transparent to the patients and that they are able to opt out of unwanted use of their data. This is particularly of concern when creating clinical dashboards such as these, as several of the data sources are used indirectly, such as when and how long a patient is online. A further risk consideration could be to ensure that the therapists understand the dashboards so they, for example, do not make conclusions on the wrong assumptions. Risk analysis can be enlightened by further user testing, although that is outside the scope of our work.

To deal with large amount of data is a major concern in health information systems design. In this paper, we have presented several ways to get an abstraction of the health and patient activity data. For example, the amount of communication between patient and therapist has been annotated in the form of a line inside the spider diagram. Another example is to use the hierarchical information from ICT-10 or SNOMED-CT ontology to change the level of abstraction in the presentation of the data. However, for a large number of patients this can be an issue and we must investigate further on scaling the visualization.

In [8], the authors proposed a conceptual framework to support model-based visualization system. The visualization artefacts that they proposed are capable of representing the relationship and constraints of software components. In this paper, we proposed a different technique to visualise health care information. We emphasised on utilizing a model-based approach for the construction of reusable visual components.

In [5], the authors presented a vision for the future of data-driven technology by means of incorporating IoT devices and visualizing health care information, e.g. bio-markers for monitoring patients physical condition. They urged for the need of using such technology and visualization in health care. However, they did not provide any information about the development of such a system and the reusability of software components. In this paper, we provide a case study and present reusable visual components that can be reused in various health care settings.

Keim et al. [22] provided a conceptual framework of how visualization could fit into the phases of information life cycle. They argued for the importance of relevant information and provided a definition of visual analytics. They also provided a list of technical challenges for advanced visual analytics solutions. Our work fits well into the conceptual framework of visual analytics as presented by Keim et al., and in this paper we address several challenges such as infrastructure, visual representation and level of detail, and data dimensionality.

Initially, in this paper we made reference to data-driven approaches to improving practice in the field of education and claimed relevance to the field of mental health, as both have human learning as a foundation for development and progress. The experience API (xAPI) is a learner-centric [25], standardised way to model interaction data in online learning environments [26], providing a known and structured data output from online learning activities. xAPI’s standardised mapping allows the integration of a range of services [24], pedagogical strategies [28], and analysis of student interaction data across sources [4]. Although not entirely similar, we argue that the activities of a student or teacher in an online learning environment share a lot of common ground with the activities of therapists and patients in iCBT. Furthermore, we argue that the field of iCBT would benefit from a unified model of patient activities or experience, for example, by capturing patient interaction data across devices.

In [3], the authors introduced the idea of using model-based approach for big data analysis. Their approach utilises xAPI and facilitates the easy creation reports which are common to standard learning analytics solutions. They presented a methodology data analysis steps that can be specified using declarative models. However, in their approach, authors only considered using scatter plot chart.

Streit et al. in [36] presented a model-based design approach for visual analysis of heterogeneous data from health care. They addressed the issue of how health care data from various sources can be linked and how visualization can be used for supporting investigation. They applied their design process to a biomedical use case where they considered visualizing medical data consisting of MR/CT/X-ray, gene/protein expression, laboratory results, disease database, etc. While our work overlaps with their approach in many aspects, in our work we focused on the mental health care domain and emphasised on constructing visualization that provides meaningful information for therapists providing Internet-based treatment. The visualization of data incorporated with an ontology as presented in our approach will facilitate health care workers to investigate data from various level of abstraction.

In [12], Medeiros et al. presented an ontology-based process mining approach where they discussed the necessity of relating elements in event logs with their semantic concepts. In their approach, they linked event logs with the concepts from an ontology which enabled them to perform concept-based analysis. The idea of using semantics makes it possible to automatically reason or infer relationships of concepts that are related. They distinguished between the application of process mining in two different levels: instance level and conceptual level. They illustrated an example process model to repair telephones in a company. The process model includes three different ontologies: task ontology, role ontology and performer ontology. The idea of using an ontology for process mining presented in [12] is very similar to our approach. The idea of filtering based on ontological concepts and the idea of grouping nodes by a high-level ontological concept is similar. However, in our approach we emphasise on various kinds of data visualization where ontology plays a major role for providing various levels of abstraction for health care information. While in [12] the authors implemented their technique in ProM, our approach is more general and can be plugged into several areas in the health care system.

Grieg [16] et al. presented an architecture for accessing health care data using HL7 FHIR and provided a methodology for visualizing health care information in various ways. Their visualization technique includes a visualization of clinical observations and self-screening results for individual patients and/or a group of patients. A spider chart was introduced for visualizing MADRS score of a patient which shows the progression of the symptoms in a single visualization. In their work, the authors provided an evaluation of the performance of accessing health care information using HL7 FHIR API. They pointed out the fact that such architecture based on HL7 FHIR APIs may have scalability problems as HL7 FHIR consists of lot of meta-data information. In our approach, we provided an architecture which is more robust in a sense that visualizations are tied to a model. Health care information from a variety of sources can be transformed into appropriate format for visualization.

7 Conclusion and future work

In this paper, we have presented a study of user-centred development of dashboards on patient activity for therapists involved in guided iCBT. We proposed to use model-based approach for developing reusable visualization components. We presented a conceptual framework that integrates the concept of model transformation from health information system to domain models; transformation of domain models to facilitate multiple perspective for data visualization; integrated system for visualizing patients progression of symptoms and conditions. By taking a model-based approach, we aimed to create visualisations and dashboards that have value beyond our particular case study. We have involved therapists closely in the design and development process, and also in the evaluation of the results. The study has produced prototypes that could hold promise in future clinical practice within guided iCBT. One required step before implementation is further testing of the effects that the dashboards have on the clinical practice.

In current practice of Internet-based cognitive behaviour therapy, modules are designed to address several symptoms at a time. Typically, iCBT treatment modules consist of a variety of tasks. In current practice, the treatment protocols are designed and implemented to follow a sequential flow of treatment processes. In a sequential flow, there are limited options for customization. This practice introduces problems and challenges for many patients as they either need to cope with the sequential flow as defined in the treatment protocol or drop out from the Internet-based treatment options. In order to achieve usable systems for iCBT, it is essential to articulate iCBT treatment modules and interfaces containing appropriate feedback on symptoms and patient conditions. Psychologists need to know about the patients’ conditions and symptoms, and most importantly they need to foresee patients’ preferences and reactions. To support the need of psychologists, we see the potential for using visualization. However, the process of developing effective visualization is not a straightforward process. By going beyond conferring with therapists in the development, and incorporating their participation closely in the development process, we can achieve systems containing visualisations that have value in a clinical setting. Data mining techniques can be integrated into our proposed visualization engine to discover the causal relationship of modules and tasks with patients’ self-assessment scores and symptoms will allow us to develop adaptive treatment modules.