Abstract
This paper addresses the challenges and solutions in developing a holistic prevention mobile health application (mHealth app) for Germany’s healthcare sector. Despite Germany’s lag in healthcare digitalization, the app aims to enhance primary prevention in physical activity, nutrition, and stress management. A significant focus is on user participation and usability to counter the prevalent issue of user attrition in mHealth applications, as described by Eysenbach’s ‘law of attrition’. The development process, conducted in a scientific and university context, faces constraints like limited budgets and external service providers. The study firstly presents the structure and functionality of the app for people with statutory health insurance in Germany and secondly the implementation of user participation through a usability study. User participation is executed via usability tests, particularly the think-aloud method, where users verbalize their thoughts while using the app. This approach has proven effective in identifying and resolving usability issues, although some user feedback could not be implemented due to cost-benefit considerations. The implementation of this study into the development process was able to show that user participation, facilitated by methods like think-aloud, is vital for developing mHealth apps. Especially in health prevention, where long-term engagement is a challenge. The findings highlight the importance of allocating time and resources for user participation in the development of mHealth applications.
Similar content being viewed by others
Avoid common mistakes on your manuscript.
Introduction
Germany lags far behind comparably developed welfare states when it comes to digitalization of the healthcare sector [1]. For this reason, the current government wants to push ahead with digitalization [2]. Responsibility however for the digitalization lies largely with the service providers and healthcare insurances, which enjoy particular autonomy in Germany due to their self-governance rights [1]. In addition, a shortage of skilled workers and an ageing population are forcing those responsible to take action [3, 4]. Holistic preventive healthcare must play an important role in solving these problems in order to relieve the burden on the healthcare sector in long term through primary prevention. Digital resources like mobile health applications (mHealth apps) are necessary to relieve and expand the conventional health care system and play a part in this primary prevention [5, 6]. On behalf of a German health insurance company an attempt is being made to develop a holistic prevention app, the Digital Health Companion (DHC), in a multidisciplinary team. The DHC aims to help people regarding a health-oriented lifestyle in the areas of physical activity, nutrition, and stress management. According to the prevention guidelines from Germany, these are three of the four most important areas in terms of sustainable prevention [7]. The presentation of the app in a scientific context and its functionality will therefore form part of this paper.
However, mHealth applications are facing the extreme challenge of user attrition. Eysenbach coined the term ‘law of attrition’ here [8]. The rule states that after a certain period of time, only so called ‘hardcore users’ regularly take part in mHealth measures. These make up only a small percentage of all users. The majority no longer uses mHealth after a while due to relatively low motivation or face usability problems [8]. To ensure high effectiveness, this drop-out rate should be as low as possible. In addition to a team of experts, it is also essential to involve potential users in the development process of an mHealth application. This is referred to as user participation in the development process [9]. User participation can have positive influence on usage behavior and can therefore be a positive part of the whole development process [10]. This paper presents user participation in form of a usability study [10, 11] as well as a short questionnaire on intrinsic motivation of using the app [12].
The development process takes place in a scientific and university context and is therefore, as so often, confronted with major obstacles such as limited budget or external service providers. This paper therefore aims for two objectives: Firstly, the structure and functionality of an mHealth application for the German healthcare sector is presented. Secondly, the implementation of user participation in the form of a usability study is explained and placed in the context of the overall development in a multidisciplinary team. To this end, the concept of user participation and the meaning of usability in this context will be explained in more detail. The development process and the functionality of the app are then presented, followed by the usability study. Finally, the results are put into context with the development process and possible conflicts between the developers’ (experts) and users’ perspectives are discussed.
User Participation and Usability
As stated, the loss of users is one of the biggest hurdles for mHealth applications [8]. In addition to a lack of relatively low user motivation, usability problems are particularly common reasons [13]. To minimize these, user participation is becoming increasingly important in the development process [14]. Studies show that involving users in the development process of mHealth applications is crucial to ensure that they meet the needs and expectations of end users and thus achieve greater acceptance and effectiveness [15, 16].
A possibility to involve users are usability tests during the app development process [10, 12]. In the critically important field of health prevention, where engaging the target population poses significant challenges [17, 18], testing and refinment of usability is paramount [19]. The high contextuality of mHealth interventions prevents the exact transferability of results from comparable studies in this topic and must be carried out individually [19, 20] Consequently, usability testing, regardless of the results, should be considered valuable [21, 22] and involving the potential target group in the development process can increase acceptance of mHealth apps [19, 23]. No matter how well informed experts are about a certain topic, the integration of the target group into the development process is inevitable [12]. Various studies have already shown that usability testing of mHealth apps can contribute not only to an improvement of usability itself, but also in content of the app [22, 24]. As in comparable studies, the think-aloud method (TA) was mainly used here to investigate usability [12]. In this method, users speak their thoughts out loud at any time during app usage and are recorded. The video and audio materials are then analyzed.
Methods
This section first presents the basic development process and the main functions of the DHC. Then the implementation of the usability study as part of the user participation is described.
Development Process and Application Features
Getting more people engaged in an active lifestyle through health prevention can be difficult. Demands and requirements differ between target groups [25, 26]. Barriers like lack of motivation to make lifestyle changes or lack of an overview of programs can also make an implementation more difficult [26]. An app must convince not only through its content, but also through its structure and interface. The DHC was therefore developed from a team of experts in the fields of physical activity, nutrition and stress management with the support of UX designers as well as programmers. The app is designed to make health prevention as low-threshold as possible, independent of users’ time and location. To this end, the first impression of the app must be kept very simple [27]. Users should receive an individual result after answering only a few questions concerning their health. Therefore, based on existing questionnaires, four multiple choice questions were developed for each of the three prevention areas using standardized questionnaires from research. These questionnaires were the EHIS-PAQ [28] and GPAQ [29] (physical activity), the Intuitive Eating Scale-2 [30] (nutrition) and PSQ-20 (stress management) [31]. The resulting questionnaires, the so-called Quick Checks (QC), used in the DHC can be found in supplementary material 1. Through the results of this entry test users set a first health goal, which they then specify with the help of a digital coach (chatbot).
After completing the QC, users can choose between a maximum of four individual health goals to implement health prevention into daily life. The selection of goals depends on the answers in the QC. Additionally, a short summery of given answers is created. This makes it easier for the user to set their answers in context and can help to form an intention, i.e. to change something about their own health behavior [32]. After this, the login takes place. With support of a chatbot called ‘Ben’, the user refines the personal goal to match it exactly to the daily routine, for example by choosing weekdays for a push-up reminder. Based on the social cognitive theory, these specific goals are intended to help users change their own health behavior in long term [33]. As a support to the user, different media content (videos, podcasts, blogs, GIFs) is made available in personal library. Interim goals such as small exercises at the workplace are transferred into the app-calendar. An overview of the schematic structure and features of the app is provided by Fig. 1.
Screenshots of the app’s main functions are additionally shown in Fig. 2. Since the app is currently only available in German, the free translations into English for all three different quick checks can also be obtained in Supplementary material 1.
The chatbot can also be consulted on prevention issues at any time. In further steps of the technical development, the users’ questions will be answered by health experts in the first time period after the release of the app. The experts’ answers are then programmed as automatic conversations and gradually integrated into the app. The DHC should thus offer the user maximum personal benefit and be integrated into everyday life as low threshold as possible.
To achieve this goal, the operation and usability of the app must be made as simple as possible. Therefore, evaluating usability among the target audience is vital to the app’s success.
Recruitment and Participants for the Usability Test
Participants in this study had to have statutory health insurance in Germany, as the app was developed specifically for these people. They were recruited via personal contacts and did not receive any incentives for participating. A sample with good diversity in terms of age and educational status was selected to correspond approximately to insured persons of the commissioning health insurance. Eleven people (seven male) participated in this usability study. Average age was 45.55 ± 18.44 (min. 22; max. 83) years. Five participants had university degrees, two a high school diploma and four middle maturity or lower.
Materials
Basically, the participants could use all functions of the app. However, the chatbot could not yet respond to new questions from the participants and some media content as well as some graphics were still missing. In addition, various bugs occurred and there could be isolated crashes. For using the app all participant used the same iPhone provided by the developer which they could hold freely in their hands. The smartphone was connected to a notebook and screen as well as participants voice were recorded.
Questionnaires on personal data, usability and intrinsic motivation to use the app were answered digitally.
Regarding usability the System Usability Scale (SUS) [34] was used. A short form of the Intrinsic Motivation scale (IMI) for measurement of intrinsic motivation [35].
The full questionnaire in German and freely translated to English can be found in Supplementary material 2.
Design
At start, participants were informed on the general process of the study and contents of the app and recording of their voice. They filled out the questionnaire on demographics, personal experience and general usage of mHealth applications. After that, TA was explained and participants performed a practice task using a weather app like in comparable studies [12]. Participants were informed that the experimenter would only interrupt if a pause in speech lasted more than five seconds [36]. To use the app, participants were given a list of eleven tasks in total, which they had to complete. The list can be found in Supplementary material 3.
Required time for each task was measured. During the tasks, participants were asked to share their thoughts about the individual tasks, but also about the app interface [24]. After the last task, participants answered the SUS and IMI.
Data Collection and Analysis
Data collection and analysis was guided by similar studies [12, 24, 37, 38]. After completion of the test each task was rated from 0 to 3 [37]:
-
0: Task was not completed.
-
1: Task was completed with physical assistance.
-
2: Task was completed with verbal assistance.
-
3: Task was completed without assistance.
The usability problems encountered while using the app were first recorded, categorized and then rated by the investigators according to their severity using the Nielsen severity scale [39] (Table 1).
Watching and listening multiple times to screen and audio recordings, author A and a research assistant extracted qualitative comments. The evaluation was carried out independently and blinded. Discrepancies were discussed together in the final evaluation step. If no agreement could be reached, author C was consulted. The user comments were then related either to perceived usefulness [24] or the individual tasks [37].
Descriptive Results of the Usability Tests
On a five-point scale participants rated their familiarity with mHealth apps on average with 2.0 and described the frequency of use as ‘infrequent’.
A total of 103 usability problems were registered, 9.36 ± 4.12 per participant. These usability problems were sorted by category (Fig. 3):
Chronologically sorted by the structure of the app for a user while using it, the following Fig. 4 shows the parts of the app where the problems occurred:
Registered usability problems were rated according to Nielsen severity scale (Table 1):
Examples for a rating of 4 were no suitable selection option in the QC or no indication to swipe left to see more options regarding user’s body shape.
Task Performance
To complete each of the tasks, participants on average needed 17.7 ± 4.44 (min: 11.01; max: 26.97) minutes. Detailed information here can be found in supplementary material 3.
The participants’ performance for each of the eleven tasks is presented in Fig. 5:
System Usability Scale and Intrinsic Motivation
The SUS consists of ten questions with a rating from 1 (very low usability) to 5 (very high usability) for each question (supplementary material 2). A conversion results in an overall rating between 0 and 100 points. A final rating above 68 points is considered above average and a fine usability [34]. On the SUS in this study, participants on average rated the usability 82.05 ± 11.77 (median: 85.00; min: 62.50; max: 97.50).
The short form of the IMI was used for self-reported evaluation of how much the participants valued the DHC. Seven questions were answered using a seven-point scale. The mean IMI score was 5.06 ± 1.51 (median: 5.57; min: 3.00; max: 7.00).
Connections and Correlations
Table 2 gives an overview of connections and correlations (Spearman) between different factors measured in this study.
Qualitative Assessment
Overall, data analysis revealed a certain discrepancy between the perceived performance of the app and participants’ verbal statements. Despite the usability problems and difficulties with the tasks, participants on average were very positive about the app. They reported the app was easy to handle, simple to use and clearly designed. This discrepancy is particularly noticeable in task completion. None of the participants expressed problems with the tasks, while some of them did not complete them.
An example is task 7, where participants were asked to change their password. To change the password, users must press the edit button for personal data in the profile screen. This task had the lowest completion score. Participants’ sample sentences were for example: ‘I don’t know what to do here, I only see a button for logout’ or ‘I must be wrong here, I will go to the home screen’. After the edit button was shown to them, sentences like ‘Ah, there it is!’ were made. Additionally, a participant who changed password without problems said: ‘I suppose that many have a problem finding this button.’
Still no participant mentioned problems or difficulties after the final task completion in overall feedback but commended various features of the app.
Discussion and Benefits of User Participation
Principal Findings
In this study, TA with task completion was used to analyze usability of a self-developed holistic health prevention app for user participation in the development process by a multidisciplinary team. Eleven participants were asked to complete eleven tasks in the app. Overall, 103 usability problems were reported of which nine were rated as usability catastrophe, 27 as a major and 49 as minor problems. Most problems were related to terminology, functionality, design, and QC. Regarding contents of the app, 39 problems were reported doing QC and 14 while choosing individual health goals. These results are integrated into the development process to improve the quality of the app and adapt it to the needs of potential users.
Overall participants rated the usability of the app highly in their statements and in responding to the SUS. Regarding participants’ task completion, a significant correlation was found for self-reported usage of mHealth apps and a negative correlation for participants’ age.
App Customization Through User Participation
The aim of the DHC is to provide users with an easy and short introduction and to set an initial individual health goal after just a few questions (QC). These questions are based on the content of scientific health questionnaires (see supplementary material 1). Scientific questionnaires were used as a basic framework. In order for users to be more likely to answer these questions, they had to be shortened considerably [40]. Shortening can have an enormous impact on the significance and can cause problems [41, 42]. Next to scientific quality criteria important information can be lost or the questionnaire may not cover all relevant aspects of a specific topic.
Problems of this type were also observed with striking frequency in this study. Nearly 36.8% of all usability problems occurred during QC, which was only one of the tasks. Since QC is the first function and has a decisive influence on the first overall impression [43], working on the problems is of high priority. Problems related mostly to terminology and choices. In terms of understanding, one user did not understand an answer option (‘What is brain exercises’). In the further process, alternatives were developed for the answer option ‘brain exercise’. The new options included ‘gaming’, ‘puzzles’ or ‘active relaxation’ and must be evaluated again. One did not fully understand a question: ‘Does this question refer only to my private life or also to my work?’ As a result, a short note was added to each question if it related exclusively to work or leisure time. Also the fifths response option for nutrition was not perceived which resulted in non-sufficient answers in some cases: ‘For me none of the answers fit.’ or ‘I think all options are bad for me.’ The reason was that only four answer options were visible on the screen and you had to scroll down for the fifth. As seen in Fig. 2 (top left), this was fixed by a programmer. As mentioned, changing the password (task 7) was a problem for some participants. The symbol (pen writing on paper) was then significantly enlarged and placed more centrally.
A very important part of the app is the individually displayed content for each user (videos, audios, GIFs). As the scripts were written by experts in the respective field with the support of a content producer, it was very important to review this content by potential users. All content played out was perceived as understandable and appropriate by the users. One user recommended videos with real people instead of animated ones. This point leads to a problem regarding user participation in the development process.
Involving potential user into the development of an mHealth app as early as possible is important and brings many benefits [44], but also conflicts can occur here. One example is the conflict between user perspective and evidence [45]. Regarding the DHC, it is not possible to just use recommendations for the QC from participants. The funder claims that this app is based on scientific instruments. The answers in the QC must therefore at least always be based on existing questionnaires or be similar. In addition, the costs and benefits must always be weighed up during the development. Users do not have to take this into account [46]. This can lead to problems, especially for such a project with a limited budget in a scientific context. One example of this study concerns the font size of the app, which was rated as too small in some cases. Since a change in the font size by the programmer would have required partial changes to the design, a new budget for UX designers would have been necessary. This was no longer planned at that time. In such cases, the costs and benefits must be weighed up on individual basis. In this case, the decision was made not to make changes, as only one person commented on this and a change in font size can also be set individually depending on the smartphone.
This shows how important user participation is in a development process and, depending on the circumstances, as early as possible. However, this feedback should never be accepted blindly. Various factors such as evidence or costs/benefits must always be weighed up.
Demand Characteristics in Think Aloud and Further Limitations
Several differences occurred regarding the participants’ positive awareness of the app and the number of usability problems. One reason could be the principle of demand characteristics [47]. In this TA study, the experimenter sat directly behind the participants. The participants were also explained that the experimenter would intervene after five seconds of silence. In order to make the experimenter feel as positive as possible regarding the app, it is possible that the participants rated the app rather more positively at the end. Furthermore, there could be difference in the general impression of the app and the processing of the individual tasks. Participants may not be aware of the usability problems expressed or may not rate them as particularly serious themselves.
The number of participants appears small, but is reasonable compared to similar studies [12, 37]. Participants had to have statutory health insurance in Germany and an attempt was made to design the test group as heterogenous as possible regarding different criteria like age, gender or educational status. Furthermore, it can be assumed that three quarters of all usability problems of an application can already be found by testing with five participants [48].
The experience of participants in using mHealth apps is generally considered to be rather low. The possible unbiasedness of the subjects may be a strength of the study in this regard. On the other hand, hardly any assessments are possible by the participants regarding previous experiences with other apps.
In addition, there are general limitations of TA. It is based on the assumption that participants can verbalize the contents of their working memory [49]. This situation is quite unusual for people. The pressure of acting and talking at the same time can cause unwanted statements and reactions from the participants [50].
Conclusions
User participation is a must have in mHealth development. Communication of health issues is difficult and user feedback is therefore all the more crucial. This paper presents the development and functionality of a holistic prevention app for people with statutory health insurance in Germany. The usability of this app was tested using TA at an early stage of development. Important problems such as the clarity of symbols or the comprehensibility of technical terms were identified regarding usability and solved. However, a few comments could not be implemented. Weighing up evidence and costs/benefits, especially in the area of health prevention, is very important. In addition, TA has proven to be a profitable method for identifying usability problems. Time and personnel resources for user participation should therefore be included in the calculation of mHealth apps before and during development. For mHealth, where only ‘hardcore users’ use a product in the long term, user centricity is one of the most important success factors.
Data Availability
The data that support the findings of this study are available from the corresponding author upon reasonable request.
Abbreviations
- DHC:
-
Digital Health Companion (the developed mHealth application)
- IMI:
-
Intrinsic Motivation Inventory
- mHealth:
-
Mobile Health
- QC:
-
Quick Check (DHC introduction test)
- SUS:
-
System Usability Scale
- TA:
-
Think aloud method
References
Behm ME, Klenk T. Digitalisierung im Gesundheitssektor. In: Klenk T, Nullmeier F, Wewer G, editors. Handbuch Digitalisierung in Staat und Verwaltung. Wiesbaden: Springer VS; 2020. p. 1–12.
Hommel T. Lauterbach: Startschuss für eRezept noch in diesem Jahr. InFo Hämatol Onkol 2022;25(6):54. https://doi.org/10.1007/s15004-022-9094-3
Reiff E, Gade C, Böhlich S. Handling the shortage of nurses in Germany: Opportunities and challenges of recruiting nursing staff from abroad: Bad Honnef: IUBH Internationale Hochschule; 2020. IUBH Discussion Papers - Human Resources; 3/2020.
Abbasi S. Desperation of Demographic Change in the Health Sector in Germany: Necessary Strategy Orientation for Medical Practices in the Countryside. UCPA 2019;16(1). https://doi.org/10.17265/1548-6591/2019.01.002
Risling T, Martinez J, Young J, Thorp-Froslie N. Evaluating Patient Empowerment in Association With eHealth Technology: Scoping Review. J Med Internet Res 2017;19(9):e329. PMID:28963090
Tossaint-Schoenmakers R, Versluis A, Chavannes N, Talboom-Kamp E, Kasteleyn M. The Challenge of Integrating eHealth Into Health Care: Systematic Literature Review of the Donabedian Model of Structure, Process, and Outcome. J Med Internet Res 2021;23(5):e27180. PMID:33970123
GKV-Spitzenverband. Leitfaden Prävention: Handlungsfelder und Kriterien nach § 20 Abs. 2 SGB V; 2021.
Eysenbach G. The law of attrition. J Med Internet Res 2005;7(1):e11. PMID:15829473
Kylberg M, Haak M, Iwarsson S. Research with and about user participation: potentials and challenges. Aging Clin Exp Res 2018;30(1):105–108. PMID:28361466
Beutter CNL, Zeller K, Martens UM, Pfleiderer B, Fegeler C. User-Centered Development of a Mobile App to Assess the Quality of Life of Patients With Cancer: Iterative Investigation and Usability Testing. JMIR Cancer 2023;9:e44985. PMID:37751285
Baek H, Suh J-W, Kang S-H, Kang S, Lim TH, Hwang H, Yoo S. Enhancing User Experience Through User Study: Design of an mHealth Tool for Self-Management and Care Engagement of Cardiovascular Disease Patients. JMIR Cardio 2018;2(1):e3. PMID:31758783
van Beukering M, Velu A, van den Berg L, Kok M, Mol BW, Frings-Dresen M, Leeuw R de, van der Post J, Peute L. Usability and Usefulness of a Mobile Health App for Pregnancy-Related Work Advice: Mixed-Methods Approach. JMIR Mhealth Uhealth 2019;7(5):e11442. PMID:31094353
Chan AHY, Honey MLL. User perceptions of mobile digital apps for mental health: Acceptability and usability - An integrative review. J Psychiatr Ment Health Nurs 2022;29(1):147–168. PMID:33604946
Krmpotic K, Gallant JR, Zufelt K, Zuijdwijk C. User-centred development of an mHealth app for youth with type 1 diabetes: the challenge of operationalizing desired features and feasibility of offering financial incentives. Health Technol. 2022;12(2):499–513. https://doi.org/10.1007/s12553-022-00656-9
Chudyk AM, Ragheb S, Kent D, Duhamel TA, Hyra C, Dave MG, Arora RC, Schultz AS. Patient Engagement in the Design of a Mobile Health App That Supports Enhanced Recovery Protocols for Cardiac Surgery: Development Study. JMIR Perioper Med 2021;4(2):e26597. PMID:34851299
Ravn Jakobsen P, Hermann AP, Søndergaard J, Wiil UK, Clemensen J. Development of an mHealth Application for Women Newly Diagnosed with Osteoporosis without Preceding Fractures: A Participatory Design Approach. Int J Environ Res Public Health 2018;15(2). PMID:29438343
Naidoo J, Wills J. Lehrbuch Gesundheitsförderung. 3., aktualisierte Auflage, deutsche Ausgabe. Bern, München: Hogrefe; Ciando; 2019. ISBN: 9783456957449.
Hardcastle SJ, Hancox J, Hattar A, Maxwell-Smith C, Thøgersen-Ntoumani C, Hagger MS. Motivating the unmotivated: how can health behavior be changed in those unwilling to change? Front Psychol 2015;6:835. PMID:26136716
Liew MS, Zhang J, See J, Ong YL. Usability Challenges for Health and Wellness Mobile Apps: Mixed-Methods Study Among mHealth Experts and Consumers. JMIR Mhealth Uhealth 2019;7(1):e12160. PMID:30698528
Enam A, Torres-Bonilla J, Eriksson H. Evidence-Based Evaluation of eHealth Interventions: Systematic Literature Review. J Med Internet Res 2018;20(11):e10971. PMID:30470678
Simons D, Bourdeaudhuij I de, Clarys P, Cocker K de, Vandelanotte C, Deforche B. A Smartphone App to Promote an Active Lifestyle in Lower-Educated Working Young Adults: Development, Usability, Acceptability, and Feasibility Study. JMIR Mhealth Uhealth 2018;6(2):e44. PMID:29463491
Mazaheri Habibi MR, Khajouei R, Eslami S, Jangi M, Ghalibaf AK, Zangouei S. Usability testing of bed information management system: A think-aloud method. J Adv Pharm Technol Res 2018;9(4):153–157. PMID:30637234
Nitsch M, Dimopoulos CN, Flaschberger E, Saffran K, Kruger JF, Garlock L, Wilfley DE, Taylor CB, Jones M. A Guided Online and Mobile Self-Help Program for Individuals With Eating Disorders: An Iterative Engagement and Usability Study. J Med Internet Res 2016;18(1):e7. PMID:26753539
Beauchemin M, Gradilla M, Baik D, Cho H, Schnall R. A Multi-step Usability Evaluation of a Self-Management App to Support Medication Adherence in Persons Living with HIV. Int J Med Inform 2019;122:37–44. PMID:30623782
Liljas AEM, Walters K, Jovicic A, Iliffe S, Manthorpe J, Goodman C, Kharicha K. Engaging ‘hard to reach’ groups in health promotion: the views of older people and professionals from a qualitative study in England. BMC Public Health 2019;19(1):629. PMID:31122239
Geense WW, van de Glind IM, Visscher TLS, van Achterberg T. Barriers, facilitators and attitudes influencing health promotion activities in general practice: an explorative pilot study. BMC Fam Pract 2013;14(1):20. PMID:23394162
Aliaksei Miniukovich, Antonella De Angeli. Visual Impressions of Mobile App Interfaces; 2014.
Finger JD, Tafforeau J, Gisle L, Oja L, Ziese T, Thelen J, Mensink GBM, Lange C. Development of the European Health Interview Survey - Physical Activity Questionnaire (EHIS-PAQ) to monitor physical activity in the European Union. Arch Public Health 2015;73:59. PMID:26634120
Armstrong T, Bull F. Development of the World Health Organization Global Physical Activity Questionnaire (GPAQ). J Public Health 2006;14(2):66–70. https://doi.org/10.1007/s10389-006-0024-x
Ruzanska UA, Warschburger P. Psychometric evaluation of the German version of the Intuitive Eating Scale-2 in a community sample. Appetite 2017;117:126–134. PMID:28645751
Fliege H, Rose M, Arck P, Levenstein S, Klapp BF. PSQ - Perceived Stress Questionnaire; 2009.
Lippke S, Renneberg B. Theorien und Modelle des Gesundheitsverhaltens. In: Renneberg B, Hammelstein P, editors. Gesundheitspsychologie. Berlin, Heidelberg: Springer Berlin Heidelberg; 2006. p. 35–60.
Bandura A. Health promotion by social cognitive means. Health Educ Behav 2004;31(2):143–164. PMID:15090118
Jordan PW. Usability Evaluation in Industry. Boca Raton: Chapman and Hall/CRC; 2014. ISBN: 9781498710411.
Ryan RM, Deci EL. Self-determination theory and the facilitation of intrinsic motivation, social development, and well-being. Am Psychol 2000;55(1):68–78. PMID:11392867
Ericsson KA, Simon HA. Verbal reports as data. Psychological Review 1980;87(3):215–251. https://doi.org/10.1037/0033-295X.87.3.215
Storm M, Fjellså HMH, Skjærpe JN, Myers AL, Bartels SJ, Fortuna KL. Usability Testing of a Mobile Health Application for Self-Management of Serious Mental Illness in a Norwegian Community Mental Health Setting. Int J Environ Res Public Health 2021;18(16). PMID:34444416
Georgsson M, Staggers N. An evaluation of patients’ experienced usability of a diabetes mHealth system using a multi-method approach. J Biomed Inform 2016;59:115–129. PMID:26639894
Nielsen J, Mack RL. Usability Engineering: Elsevier; 1993. ISBN: 9780125184069.
Sahlqvist S, Song Y, Bull F, Adams E, Preston J, Ogilvie D. Effect of questionnaire length, personalisation and reminder type on response rate to a complex postal survey: randomised controlled trial. BMC Med Res Methodol 2011;11:62. PMID:21548947
Beebe TJ, Rey E, Ziegenfuss JY, Jenkins S, Lackore K, Talley NJ, Locke RG. Shortening a survey and using alternative forms of prenotification: impact on response rate and quality. BMC Med Res Methodol 2010;10:50. PMID:20529365
Kost RG, Rosa JC de. Impact of survey length and compensation on validity, reliability, and sample characteristics for Ultrashort-, Short-, and Long-Research Participant Perception Surveys. J Clin Transl Sci 2018;2(1):31–37. PMID:30393572
Chiu W, Cho H, Chi CG. Consumers’ continuance intention to use fitness and health apps: an integration of the expectation–confirmation model and investment model. ITP 2021;34(3):978–998. https://doi.org/10.1108/ITP-09-2019-0463
Marent B, Henwood F, Darking M. Development of an mHealth platform for HIV Care: Gathering User Perspectives Through Co-Design Workshops and Interviews. JMIR Mhealth Uhealth 2018;6(10):e184. PMID:30339132
Jahnel T, Schüz B. Partizipative Entwicklung von Digital-Public-Health-Anwendungen: Spannungsfeld zwischen Nutzer*innenperspektive und Evidenzbasierung. [Participatory development of digital public health: tension between user perspectives and evidence]. Bundesgesundheitsblatt Gesundheitsforschung Gesundheitsschutz 2020;63(2):153–159. PMID:31915866
Tan JPY, Tan MWJ, Towle RM, Lee JSW, Lei X, Liu Y, Goh RSM, Chee Ping FT, Tan TC, Ting DSW, Lee CE, Low LL. mHealth App to Facilitate Remote Care for Patients With COVID-19: Rapid Development of the DrCovid + App. JMIR Form Res 2023;7:e38555. PMID:36649223
McCambridge J, Bruin M de, Witton J. The effects of demand characteristics on research participant behaviours in non-laboratory settings: a systematic review. PLoS One 2012;7(6):e39116. PMID:22723942
Nielsen J, Landauer TK. A mathematical model of the finding of usability problems. In: Arnold B, editor. Proceedings of the INTERACT ‘93 and CHI ‘93 Conference on Human Factors in Computing Systems. New York, NY: ACM; 1993. ISBN:0897915755. p. 206–213.
Monsalve B, Aliane N, Puertas E, Andrés JF. Think Aloud Protocol Applied in Naturalistic Driving for Driving Rules Generation. Sensors (Basel) 2020;20(23). PMID:33287222
Nitsch M, Adamcik T, Kuso S, Zeiler M, Waldherr K. Usability and Engagement Evaluation of an Unguided Online Program for Promoting a Healthy Lifestyle and Reducing the Risk for Eating Disorders and Obesity in the School Setting. Nutrients 2019;11(4). PMID:30934754
Acknowledgements
Special thanks to Klara Rinne for her support in conducting the study and analyzing the data.
Funding
Open Access funding enabled and organized by Project Deal.
Open Access funding enabled and organized by Projekt DEAL.
Author information
Authors and Affiliations
Contributions
All authors have planned the study. P.B. and C.G. analyzed the data from the study. P.B. and C.G. wrote the main manuscript text. P.B. prepared all figures and tables. I.F. and C.G. reviewed the manuscript. P.B. reworked the manuscript.
Corresponding author
Ethics declarations
Ethical Approval
The study was approved by the ethical committee of the German Sport University Cologne (reference: 098/2022).
Competing Interests
The authors declare no competing interests.
Additional information
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Electronic Supplementary Material
Below is the link to the electronic supplementary material.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Bickmann, P., Froböse, I. & Grieben, C. An mHealth Application in German Health Care System: Importance of User Participation in the Development Process. J Med Syst 48, 20 (2024). https://doi.org/10.1007/s10916-024-02042-6
Received:
Accepted:
Published:
DOI: https://doi.org/10.1007/s10916-024-02042-6