Keywords

1 Introduction

Healthcare remains focused on disease management, and not on looking holistically at the health and wellbeing of the whole person [1]. The latter approach aims at empowering patients by helping them better manage their health [2,3,4]. Heath literacy/education, information-seeking behavior, and shared decision making characterize an empowered person who understands his/her health/conditions and communicate with medical professionals to get care that is compatible with his/her needs, values, and best interests [5,6,7]. To be empowered in this way people/patients must:

  1. 1.

    have the necessary knowledge and self-awareness to understand their conditions and treatment options;

  2. 2.

    be able to make informed and conscious health choices (i.e., decide);

  3. 3.

    actively manage, with or without advice from medical professionals, their health and well-being (i.e., act).

Few applications for patient empowerment exist [1, 5]. Most members of the public, including patients, usually obtain health information from Web searches using generic search engines [4, 8], which is often overwhelming, too generic, outdated, and of poor quality [8, 9]. Although Artificial Intelligence (AI) could play an important role in health empowerment [10, 11], it presently mostly provides self-diagnosis apps, which act as substitute doctors and keep patients as passive recipients [12,13,14]. This paper provides further details and evaluation of an Intelligent Empowering Agent (IEA) [15] that exploits the whole Internet and uses AI to empower people/patients to obtain, through natural language, tailored, easily understood, and trusted health information from the Web.

2 Background and Motivation

2.1 Shortcoming of Current Available Conversational Agents for Patient Empowerment

Conversational agents are AI systems that simulate conversations with users, and inform them by generating easily understood dialogue. Only a limited number of studies have linked person/patient empowerment to AI and conversational agents. A literature review [15] found that:

  • Conversational agents are mostly created for a specific condition.

  • Empowerment is almost never addressed directly and, when it is, only some aspects are considered.

  • Comprehension of health information/conditions is the least addressed step of empowerment.

  • Information is seldom tailored to the needs of the user.

  • The origin and veracity of the provided content is often not provided and all available information (e.g., on-line) is not used.

  • User requirements, such as language complexity or information quality are not considered.

2.2 Requirements of Online Health Information Seekers

Previously published research and a literature review [4] have found that, when searching for health information on the Web, the main user requirements on the found information are:

  • language complexity;

  • information quality;

  • information classification/customization (tailoring).

Language must be easy to understand and the information provided scientifically correct [16]. Moreover, health information should be tailored to the users unique needs and interests [17]. Since tailored health information is more personally relevant, it is more likely to be read, understood, and acted on [18,19,20,21]. As a consequence, empowering health information must be easy to understand (by a non-medical expert), of good quality (trustworthy), and tailored to the user specific profile and needs [4, 22].

2.3 Research Objective

By combining the potential of AI with the vast amount of health information available on the Web, the research aims to improve user empowerment by providing tailored health information that is trustworthy and easily understood, through an Intelligent Empowering Agent (IEA) that:

  • interacts with the user to understand his/her profile and information need;

  • retrieves information from the Web and uses an AI algorithm to customize it;

  • presents this information as a tailored, intelligible, and trustworthy output that facilitates the users comprehension and decision making.

3 Principles of an Intelligent Empowering Agent

The components of an IEA are:

  1. a

    User query

    The user selects a complaint from a list or directly enters it as free text.

  2. b

    User profile

    The user profile is constructed from each user’s current health status (e.g., symptoms and/or conditions), background health status (e.g., sex, age, gait, BMI, and comorbid illness), lifestyle information (e.g., sleep, drugs, and meal composition), dynamic health indicators (e.g., vital signs monitoring, physical activity monitoring and stress level), empowerment level (e.g., health literacy, motivation and gaining control), health and well-being needs (e.g., urgent health improvement and elective quality of life).

  3. c

    Search engine

    The search engine retrieves health information from the World Wide Web, health-data repositories, and internal information coming from previous searches (anonymized).

  4. d

    AI algorithm

    The AI algorithm takes the health information collected by the search engine and organizes it into categories (e.g., complaint description, alternate names, related complaints, and related diseases) by using a machine learning algorithm that analyses the section headings of the retrieved documents. The most appropriate information for the user is then chosen by using a decision tree algorithm and according to the following criteria [18, 19, 22]:

    • Language complexity, to provide users with information that they can easily understand.

    • Information quality, to provide users with current, accurate, trustworthy, and unambiguous information.

    • Custom information, to provide users with tailored content (considering the user query and user profile).

  5. e

    Output presentation. Tailored relevant health information is provided on complaints (definitions of and related elements), diseases (definitions of and related elements), tests (descriptions of and related elements), and external information sources. A “traffic-light” color coding (i.e., red, amber, or green), that implies the need for an urgent consultation with a healthcare professional, is also provided. The overall process of the IEA is shown in Fig. 1.

Fig. 1.
figure 1

IEA process model.

Full size image

4 Implementation of the Intelligent Empowering Agent

An IEA prototype, the Conversational Health Agent for Person Empowerment (CHAPE) has been developed and can be accessed at http://cohealth.ivi.ie/chape. Initially, the user is asked to input age and sex and select a complaint from a list, which includes easily understood terms such as pain or discomfort, breathlessness, weakness or fatigue, etc. These complaints were derived from a classic textbook on symptoms [23], review of the literature, and expert opinion. Complaints are then further subdivided into more specific complaints, which include those most often associated with in-hospital death [24] and the commonest diagnoses encountered in primary care [25]. Alternatively, the user can directly type in any complaint in a free text area. Depending on the complaint selected and the user’s profile characteristics, such as age and sex, a further sub-list of possibly related complaints is presented, to help define the primary complaint more precisely (Fig. 2).

Fig. 2.
figure 2

CHAPE interface allows users to specify their complaints in an easy and natural way.

Full size image

An information window is then displayed (Fig. 3) and it contains:

  • Complaint name with a background colour (red, amber, or green), which indicates the health risk.

  • Complaint description.

  • Alternate names of the complaint.

  • Related complaints.

  • Disease(s) associated with the complaint.

  • Tests commonly used to further define the complaint.

  • Web information related to the complaint, which is reliable and reputable.

Fig. 3.
figure 3

Output window containing information about the searched element and related complaints, diseases, tests, and Web sites.

Full size image

When a related complaint, disease or test is selected a new information window is opened, which displays related complaints, diseases and tests ordered by taking into account the previous searches (Fig. 4).

Fig. 4.
figure 4

Related complaints and diseases are ordered by considering their correlation with the previous searched elements.

Full size image

The user’s profile and chosen complaint are used to get a list of Web sites by means of Google’s search engine. The web sites are then re-ranked by computing their language complexity and information quality.

Given a set of m Web pages, the language complexity of each Web page, is computed by considering the n words of the page (not considering the stop words) and computing the Word Familiarity (WF) as the number of Google results [22, 26, 27]. The language complexity index (LC) is then calculated according to the following formula (the higher the index the simpler the language):

$$LC{ } = \left( {\mathop \sum \nolimits_{k = 1}^{n} WF_{k} { } \div n) \div \begin{array}{*{20}c} {max} \\ m \\ \end{array} (\mathop \sum \nolimits_{k = 1}^{n} WF_{k} \div { }n} \right)$$
(1)

The information quality is evaluated in terms of reliability and timeliness, as discussed in [22, 28, 29], by checking the metadata in the Web pages related to the schema.org vocabulary or similar ones (Dublin Core, Open Graph Protocol, etc.). A weighting, with an arbitrary maximum value of 10, is assigned to each element according to its relative importance. CHAPE assigns more weighting (2/3rd) to reliability than to timeliness (1/3rd) according to the following:

Reliability

  • author: w1 = 1.

  • publisher: w2 = 1.5.

  • reviewedBy: w3 = 1.5.

  • recognizingAuthority w4 = 2.5 (Tot. 6.5).

Timeliness

  • dateCreated: w5 = 0.5.

  • dateModified: w6 = 1.5.

  • datePublished: w7 = 0.5.

  • lastReviewed: w8 = 1 (Tot. 3.5).

Thus, given a set of m Web pages, for each Web page CHAPE checks the presence of one or more quality information elements and sums up the related weights. The information quality index (IQ) is computed as follows:

$$IQ{ } = \mathop \sum \nolimits_{k = 1}^{8} w_{k} \div \begin{array}{*{20}c} {max} \\ m \\ \end{array} \mathop \sum \nolimits_{k = 1}^{8} w_{k} { }$$
(2)

Finally, the ranking index of the Web page is computed as follows:

$$R\, = \,\alpha \, *{\text{ LC}}\, + \,\left( {1 - \, \alpha } \right) \, *{\text{ IQ}}$$
(3)

where α allows to differently weigh the language complexity and information quality.

The Web pages with the highest-ranking indexes will appear first.

5 Evaluation of CHAPE

5.1 Subjective Tests

Subjective testing of CHAPE was performed by first-year social work students at the University of Palermo on 29th March 2022. After being invited to use CHAPE, students were asked to fill a short online survey (https://ec.europa.eu/eusurvey/runner/CHAPE_initial_2). The survey had four sections: 1. Non-sensitive user-profile information, 2. Questions on the use of CHAPE, 3. Desired additional features, and 4. Any other feedback. The survey presented statements about CHAPE whose agreement was expressed in a 1–5 Likert-type scale as follows:

$$1\, = \,strongly \, disagree, \, 2\, = \,disagree, \, 3\, = \,undecided, \, 4\, = \,agree, \, 5\, = \,strongly \, agree.$$

Overall, fifteen responses were obtained. The user profiles of the respondents were the following:

  • 14 females and 1 male; 14 respondents with an age range 19–40 and 1 respondent with an age range 41–60.

  • Medical knowledge: 6 none, 5 basic, 3 average, and 1 good.

  • Computer skills: 2 none, 8 basic, 2 average, 1 good, and 2 excellent.

Responses were assessed in terms of usability, user experience, perceived value, and potential users.

Usability:

most respondents found CHAPE interface clear, fast, and easy to use without prior knowledge. It helps in identifying health information about complaints, but it is less useful in identifying diseases and medical tests related to a complaint (Table 1).

User Experience:

most respondents could better understand their complaints and related diseases but not the medical tests (Table 2). The provided websites were considered trustworthy and the provided information was considered relevant.

Perceived Value:

most respondents found that CHAPE would encourage users to take a more active interest in their health and wellbeing but did not think it would improve their health and communication with doctors (Table 3).

Potential Users:

most respondents considered patients and/or the general public rather than medical professionals (including the social workers) were the most likely potential users (Table 4).

No specific comments were provided in terms of additional features and other feedbacks.

Table 1. Responses to “How easy to use do you find CHAPE?” question.
Full size table
Table 2. Responses to “How helpful do you find CHAPE?” question.
Full size table
Table 3. Responses to “Which of the following statements apply to you?” question.
Full size table
Table 4. Responses to “Who should use CHAPE according to you?” question.
Full size table

5.2 Objective Tests Based on Language Complexity and Quality Information

To evaluate the ability of CHAPE to provide better-quality information that is easy to understand, we re-ranked Google results according to language-complexity index (LC), information-quality index (IQ), and ranking index (R), by choosing α = 0.5 to equally weigh LC and IQ. Four of the most searched health terms in Google are two complaints: anxiety and depression and two diseases: diabetes and pneumonia. For the first ten Google results of these four terms, we computed LC, IQ, and R as shown in Table 5. Table 6 shows the re-ranking of the Google results of CHAPE using the R index. CHAPE alters considerably the ranking of all four keywords. To better show this, Table 7 presents the Hamming and Manhattan distances between the original Google ranking and the CHAPE ones. The Hamming distance indicates how many positions in the new ranking differ from the original ones, a value of 10 indicating that all the positions have changed. The Manhattan distance provides a quantitative value of the distances of the new positions with respect to the original, a value of 0 indicates complete agreement while the maximum of 50 indicates the ranking order is completed reverse. Although CHAPE is still under development, it is already able to re-rank health-related Web pages, provided by a generic search engine such as Google, so that information extracted from higher-quality and easier to understand Web pages is shown first.

Table 5. Computation of LC, IQ and R for anxiety, depression, diabetes, and pneumonia keywords.
Full size table
Table 6. Re-ranking of Google results.
Full size table
Table 7. Hamming and Manhattan distances.
Full size table

6 Conclusions

Intelligent Empowering Agents (IEA) should enable anyone anywhere, regardless of educational level or health literacy, to have instant access to health information they understand, which empowers them to decide the wisest interventions, if any, for their immediate and future well-being. We have designed and developed an IEA that behaves “intelligently” by allowing the user to input his/her profile and requirements in an easy way. The system provides a customized list of complaints, to choose from, and tailored health information that is of good-quality and easy understood. To our best knowledge, this is the first attempt to create an intelligent empowering agent that exploits the potential of AI and the vast amount of health information available on the Web to facilitate comprehension and action on general complaints/diseases.

Future work aims to redesign the user interface to be more conversational (chatbot like) and include more user profile information, such as gait, body type, nutritional status, comorbidities etc. Complaints and diseases will be associated with Concept Unique Identifiers (CUI) of the Unified Medical Language SystemTM to map them to standard terms taken from medical-term classifications such as ICD-9, ICD-10, or SNOMED. AI will be used not only to filter information gathered from the Web but also to process previously collected user information. Although user input is anonymous, users will be provided with an option to grant or withdraw informed consent to use their data. Once these improvements are made the agent is going to be tested on a wider demographic.