1 Introduction

The onset of a global pandemic has changed and is still changing the demands and needs of our health system. Reducing the number of outpatient appointments, implementing triage protocols that advise treating only urgent issues, and reorganizing caseloads to minimize patient contact are all strategies to encourage physical distance. Peer support groups and group psychotherapy sessions have either been discontinued, scaled back, or moved online. By using isolation rooms, lowering the total number of beds available, tightening admission requirements, and shortening admission duration, inpatient units have promoted physical separation (Moreno et al. 2020; Pinals et al. 2020; Starace and Ferrara 2020). As a result, clinicians have been forced to offer feasible solutions even from a distance without the possibility of having direct contact with patients. Many solutions have been thus developed to address COVID-19 diagnosis, monitoring, and management (Lukas et al. 2020; Moreno et al. 2020), as well as to improve psychological, psychosocial, and physical well-being (Chan et al. 2021; Riva et al. 2020). For example, telehealth, which includes remote video or phone conferencing, online blended or coached therapies, and self-help therapies offered through apps, was quickly adopted to fill the gaps in face-to-face care. The possibility to guarantee care without being physically present could be very advantageous in terms of time and financial resources for patients and the services. Moreover, these solutions allowed clinicians to reach frail people with restricted access to health services. This limitation is especially relevant when talking about the aging population (Mitnitski et al. 2001), mainly for older people who might present some limitations, like in transportation.

By 2050, there could be 2 billion people over 65 (Kinsella and Phillips 2005), which could significantly impact how health and social care are planned for and provided. Indeed, aging may cause motor and cognitive issues because of a decrease in physiological reserve and function, including the ability to handle long-term or sudden stresses (Colón-Emeric et al. 2013; McEwen 2016). As a result, people's susceptibility to unfavorable outcomes may rise as they get older. Further, several interconnected variables, including biological and environmental pathways, age-related chronic disease, a person's personality, emotional state, coping mechanisms, and cognitive status, can have an impact on the degree of vulnerability (de Vries et al. 2011; Walston et al. 2006). These factors can change over time, increasing the likelihood of unfavorable outcomes like falls, disability, and death (Clegg 2014). Among the most evident aging deficits are physical phenotypes, such as low grip strength, low energy, slowed walking speed, low physical activity, and/or unintentional weight loss (Fried et al. 2001). However, cognitive deficits and physical symptoms are linked and may influence each other (Delrieu et al. 2016; Kelaiditi et al. 2014; Miyamura et al. 2019).

The cognitive profile of aging is traditionally characterized by a linear decline starting in adulthood on measures of the efficacy or efficiency of processing (Salthouse 2010). Among the abilities that deteriorate with age are difficulties in memory performance, attention, and executive functions (Delrieu et al. 2016). The risk of cognitive decline could be considered when these problems are significant. Memory issues in particular may be essential for determining the likelihood of developing dementia, such as Alzheimer's disease (Albert et al. 2011; Serino et al. 2014). However, the early signs of cognitive decline are frequently overlooked because they are mistaken for the effects of normal physiological aging. Even though there are instances in which this may be the case, a timely assessment could be the best solution to identify the magnitude of the problems that distinguish physiological from pathological aging. The risk of falls, home accidents, and hospitalization could rise as a result of wasting time that could have been better spent developing a specialized program for cognitive training. Therefore, it is crucial to identify problems quickly and efficiently to stop the progression and prevent disability. However, the complex clinical care that older patients need may make it more difficult to recognize problems before they become serious. On the one hand, older people typically have complex and varied clinical conditions requiring specialized assistance not limited to a single specialist, and they require a variety of healthcare providers and appointments. These obligations frequently come at a high price in terms of time, money, and energy (Miyamura et al. 2019). On the other hand, cognitive deficits frequently emerge during neuropsychological evaluation, but people do not manifest problems in their routine life (Chaytor and Schmitter-Edgecombe 2003; Mondini et al. 2016). Thus, until clinical signs are latent, cognitive problems are not considered a primary care choice. Moreover, when the disease is still in its early stages, psychological and neuropsychological well-being is frequently listed among the needs deemed unimportant for primary care, although the need to consider 360-degree assistance of older people is well known nowadays (Bruni et al. 2022a). Given these reasons, implementing well-timed neuropsychological assessments and novel tools that are more sensitive to the early detection of deficits could represent a promising solution (Cipresso et al. 2014).

Traditionally, in neuropsychological assessment and rehabilitation, clinicians use paper and pencil tools, which are widespread but have little ecological validity and require the simultaneous presence of the patient and examiner in the same room. An increasingly important issue over the past years regards the ecological validity of neuropsychological tests, i.e., how to measure cognitive functions reliably and validly (Chaytor and Schmitter-Edgecombe 2003; Serino and Repetto 2018). Research suggests that most of these instruments have a moderate level of ecological validity for predicting everyday cognitive functioning (Chaytor and Schmitter-Edgecombe 2003); in fact, some individuals may perform as expected during a clinical examination but struggle in real-world settings. This concern is attributed to the fact that assessment tools do not accurately reflect the demands of the everyday world. Thus, focusing on new technologies might be the best way to provide more cutting-edge tools to facilitate remote telemedicine processes. In this regard, given their ability to create realistic environments in a managed and safe way, digital and extended reality technology are promising tools to enhance the accuracy of the neuropsychological assessment process (Cavedoni et al. 2022; Neguț et al. 2016; Riva and Mantovani 2014). Among these, a recent trend in the field of technology is 360-degree technology (videos and images) (Borghesi et al. 2022; Realdon et al. 2019). These spherical videos and photos are recorded by a special camera with circular fisheye that captures images from the surrounding environment. Such tools could address the concerns about the need to use more ecological instruments for memory evaluation (Serino and Repetto 2018). Users can look all over the scene as they would in real life by moving their heads to change the viewing direction and to take in everything that is going on. Given this, different points of view may be experienced depending on the recording type; if the camera is put on the recorder’s head while a video is being recorded, the user may get a first-person view of the scene. Otherwise, the camera can be placed anywhere in the scene, allowing the user to view it as an external observer (third-person perspective) (Borghesi et al. 2022). These features could be crucial for the assessment of memory, improving the procedure’s precision. The possibility to manipulate the position of the subject in space and place in a realistic virtual environment allows a considerable increase in the ecological validity of the tests and the promotion of an embodied experience (Riva et al. 2019).

Moreover, 360-degree media are a cost-effective technology, and the equipment is easy to use (Bohil et al. 2011; Mancuso et al. 2023). Additionally, they have a user-friendly design that makes them better suited for patients with mild to severe impairments who might struggle to operate more advanced devices (Mancuso et al. 2020; Realdon et al. 2019; Riva et al. 2020a, b; Sbordone 1996).

Therefore, in this study, we present ECO-MEMORY and its usability data as a novel tool to fit with this promising panorama. The cross-platform application, ECO-MEMORY, can be installed on the patients’ devices (e.g., tablets and smartphones) and guarantee an objective, standardized and remote evaluation of memory to keep patients monitored even without the need for additional visits and patient movement. The app was developed based on available material implemented by the authors in previous works (Bruni et al. 2022a, b) and created new content. Overall, ECO-MEMORY aims to provide the possibility to respect hygienic rules while saving time and money for both the National Health System, while still delivering excellent medical care.

2 Eco-memory

ECO-MEMORY is an app developed with Unity using files recorded in a real environment with the Insta 360 ONE X, an omnidirectional video camera that can record spherical photos and videos with a resolution respectively of 608 × 3040 and 5.760 × 2.880 pixels. The app was deployed for use on both Android and iOS devices (smartphones and tablets). It consists of four different parts, each addressing a specific memory task: (1) recognizing objects, (2) recognizing faces, (3) learning a path, and (4) creating an allocentric map. The choice of tasks was based on the most problems that literature revealed in patients risking dementia (Serino et al. 2014; Silverberg et al. 2011); for example, we introduced a task involving faces because some evidence suggests that the face-name association is one of the most common challenging situations in aging (Silverberg et al. 2011); as well as spatial deficits are crucial symptoms in the earliest stages of pathological aging (Serino et al. 2014). Additionally, we take inspiration from several used tests to evaluate cognitive deficit ecologically (Beschin et al. 2013; Cipresso et al. 2016) and adapted them to the 360° technologies. Each task was carried out in a different place, including a living room, an office, a gym, and a house, to provide familiar environments that users could encounter regularly. The goal was to implement the ecological validity of the assessment, creating situations that called for abilities that were as close to those encountered in daily life as possible. As a result, our application consists of a collection of tests to assess memory that could be better captured in 360-degree panorama. All the scenes are structured to be used in two different moments: the first, namely, learning, in which the user has to memorize specific features, also helped by the performance of some additional tasks, and the second, recalling, in which what was learned in the first phase has to be used to answer new requests. The recalling phase is supposed to take place 15 min after the end of the learning. The four tasks are presented in Table 1.

Table 1 The description of each task, learning and recalling phases
Full size table

In each scene, a 360-degree video and pictures allow the user to look around while performing the tasks. At any time, the interaction is provided either through buttons superimposed on the video or through a specific User Interface (Fig. 1). Instructions are given to the user in the form of written text and audio instructions before the learning and the recalling phases. To convey the message as clearly as possible while also taking into account the appropriate voice inflections and tones, audio clips of a therapist reading the instructions were recorded. Before beginning the task, the user has the chance to re-listen to the instructions again. The performance data for each scene are recorded and stored in.txt files.

Fig. 1
figure 1

Screenshots from the recalling phase of each scene: a recognizing objects, b recognizing faces, c recognizing a path, d creating an allocentric map

Full size image

3 Usability test

Usability can be defined as the degree to which a specific user can utilize a given system to accomplish specific goals within a well-defined context of use. Particularly, three elements are considered: effectiveness (the possibility that users will be able to accomplish goals), efficiency (the effort put forth by users to achieve goals), and satisfaction (users’ considerations about the interaction with the system) (Argent et al. 2018; Brooke 1996). When new tools and technologies are developed, usability is an important aspect that should be analyzed to understand human-technology interaction thoroughly. As outlined by Tuena and colleagues (Tuena et al. 2020), to evaluate the usability of an application is necessary, especially in aging, to (i) identify obstacles and facilitators, (ii) develop appropriate tasks for the sample, (iii) define the usability criteria, (iv) test its clinical use.

3.1 User experience measures

In the present study, usability has been assessed using the System Usability Scale (SUS) (Bangor et al. 2009; Brooke 1996), the Senior Technology Acceptance Model (STAM) (Chen and Lou 2020), and the Independent Television Commission Sense of Presence Inventory (ITC-SOPI) (Lessiter et al. 2001). Additionally, we used a qualitative instrument: the Thinking Aloud Protocol (TAP) (Lewis 1982) to collect information about the usability and the user experience with the technology.

SUS is a fast (10 items) and easy-to-use questionnaire created by Brook in 1996 (Brooke 1996). Users need to express the degree of the agreement in a range from 0 to 4 for each statement. The final score can range from 0 to 100: the higher the score, the better the usability (Bangor et al. 2009).

The STAM is a 13-item scale that analyzes four components of the Senior Technologies Acceptance Model (Chen and Lou 2020): beliefs, perception of control, anxiety related to technologies, and general health conditions. For each of these areas, participants must define their degree of agreement with the statements using a 10-point Likert scale ranging from “Strongly Agree” to “Strongly Disagree”.

The ITC-SOPI is a self-report questionnaire that includes 42 items addressed to investigating the individual’s feelings during the experience as referred to four subscales: Sense of Physical Space (17 items), Engagement (13 items), Ecological Validity (5 items), and Negative Effects (6 items). In our study, we considered only the Ecological Validity and Negative Effects scale. Thus, users were asked to determine their degree of agreement with each sentence using a 5-point Likert scale ranging from “Strongly Agree” to “Strongly Disagree”.

The TAP is a qualitative technique aimed to collect information about usability and interaction from the point of view of the final users. Particularly, users are requested to express their opinion regarding the technology employment and criticism while performing the task; all the users’ verbalizations are transcribed and analyzed to develop the formal usability report. The outcome is a description of the main difficulties that emerged from the user interaction, the impact of the problem on usability, and the practical solutions. The results of the analysis could be used to refine the interaction design. TAP transcriptions were divided into three primary areas: (1) use of the tablet; (2) main pages; (3) exercises.

3.2 Participants

A sample of 13 participants was enrolled among the patients and outpatients of the IRCCS—Istituto Auxologico Italiano in Milan. They were volunteers aged over or equal to 60 (without maximum age limitation) of both sexes, with normal or corrected-to-normal vision. Exclusion criteria were: (i) invalidating internist, psychiatric, and neurological conditions which could affect the performances; (ii) cognitive impairments certifiable by a score at the Mini-Mental State Examination (MMSE), Italian version (Magni et al. 1996; Measso et al. 1993), lower than 26 points. The resulting sample included 7 females and 6 males, with a mean age of 71.46 (SD = 6.01) and a mean of 12.31 (SD = 3.22) years of education. All the participants’ demographic data and MMSE scores are reported in Table 2. The study received ethical approval from the Ethical Committee of the Istituto Auxologico Italiano, and all patients signed the informed consent before being enrolled in the study.

Table 2 Demographic data and Mini-Mental State Examination (MMSE) scores
Full size table

3.3 Procedure

Participants underwent an examination in a single, roughly 1.5-h session that included four phases: (i) preliminary discussion with the user about the aim of the study and the possible negative effects of 360-degree videos and informative consensus sign; (ii) gathering participant personal data (age, sex, education, MMSE) and their confidence with technology and electronic devices; (iii) conducting ECO-MEMORY session; (iv) administering usability tests.

Before starting the activity, the experimenter trained patients on how to move in 360-degree videos on the tablet and interact with the icons via the touch screen. The assessment task then began after it was confirmed that they understood basic operations. The experimenter turned on the application and gave instructions to complete all the situations following the commands; such instructions were read to ensure all the participants received the same commands. The user was asked to comment aloud on the activities while executing tasks, as requested by the TAP.

To test usability, we ran the app on a tablet Samsung Galaxy Tab A6 (SM-P580). As prior described, the application consists of different sub-tests that analyze different sub-components of memory: a task of free recall and recognition of objects, a task of free recall and recognition of faces, and two tests to evaluate spatial memory (learning a route and learning the plan of an apartment). For each task, there is a phase in which the stimuli are presented using 360-degree media and a subsequent response phase that includes interactive response screens.

4 Results

Starting from the quantitative data, according to Bangor and colleagues (Bangor et al. 2009), the mean score of the SUS indicates that users placed the application in a marginal zone of acceptability (M = 64.6; SD = 20.1); however, it is in a portion between “Ok” and a “Good” level of usability as shown in Fig. 2.

Fig. 2
figure 2

Graphical representation of the interpretation of SUS. The mean score obtained (64.6) is indicated by the vertical line, according to the rating comparison scale provided by Bangor 2009

Full size image

The results of the STAM scale reveal that users have a positive attitude toward technology (M = 9/10; SD = 1.4), good control/access to technological devices (M = 8.2/10; SD = 1.4), have a low level of anxiety related to technology (M = 4/10; SD = 3.3), and consider themselves in good health conditions (M = 7.7/10; SD = 0.8). As shown by the ITC-SOPI sub-scale investigating ecological validity, a good level of naturalness was reported (M = 3.6/5; SD = 0.5). Referring to the sub-scale of negative effects, all participants reported a low score of side effects (M = 2/5; SD = 0.9), indicating that the use of ECO-MEMORY did not induce dizziness and cybersickness. Table 3 shows User Experience (UX) measures.

Table 3 Descriptives of the user experience measure. Mean, the maximum available score, standard deviation, and minimum and maximum are reported for each of them
Full size table

Qualitative results of the thinking-aloud protocol are shown in Table 4. Overall, participants had little difficulty using the tablet or navigating the main page of the app. Most of them reported difficulties with task 2, which involved encoding faces referred to their possible colleagues or job manager and actively recognizing them by moving around the office or tapping and selecting the names. Nine people reported difficulties in listening and comprehending the instruction due to the low quality and the complexity of the audio. Six participants mentioned issues with the video quality, such as dark or blurry images; five people reported some interaction issues with the exercise progress buttons that allowed them to switch between offices. Two people had difficulty recognizing colleagues. Similar problems were reported in the other tasks, in which participants frequently commented on the low quality of the images and had trouble exploring the environments. Moreover, several participants complained about difficulties in task 3, which required them to watch a video and recall the presented itinerary while also paying attention to some objects the experimenter put in the gym along the way. Six participants reported difficulties with task execution related to exploration, the slowness of the video, or the object perceived as unclear. One of them had some difficulties identifying the symbol associated with the object choice during the recall phase of task 3.

Table 4 Qualitative usability results of Thinking Aloud Protocol
Full size table

5 Discussion

Providing functional feedback that truly reflects the patients’ capability to respond to daily challenges requires the assessment of cognitive functions in a way that matches as closely as possible what occurs in real life. In our work, a novel tablet-based application with 360-degree media has been developed to address the issue of the ecological validity of the currently available test, which used abstract situations. ECO-MEMORY was developed using 360-degree contents to recreate real-world situations, in agreement with the ongoing scientific debates supporting the use of new technology in neuropsychological assessment (Bilder and Reise 2018; Casaletto et al. 2017).

The idea behind this application was inspired by the desire to overcome the limitations imposed during the pandemic era. This period highlights the increasing need for focused interventions that deal with a variety of previously unrecognized circumstances. Examples include the inability of patients to receive care because of their health issues or a dearth of caregivers who could help them travel. By offering a preliminary remote solution to evaluate deficits without meeting clinicians in the hospital setting, ECO-MEMORY may be able to help with these issues.

ECO-MEMORY is currently evaluated as an easy-to-use tool for patients who could interact with it independently under the supervision of clinicians. The application might include in the future a remote data sending feature in addition to an automatic scoring system. To close the gap between those in the most disadvantaged circumstances, patients may receive memory assessment remotely, taking into account their limitations. This will save time and money for the patients. According to the literature (Realdon et al. 2019; Serino and Repetto 2018), our usability data are promising. Participants found ECO-MEMORY to be “good” and “usable” and were willing to use it (Bangor et al. 2009; Chen and Lou 2020). Users were able to accomplish all expected goals without learning a lot of processes or complex actions, as highlighted by low scores on some questions in the SUS questionnaire (e.g., I needed to learn many processes before successful interact with the app). This is in line with two core features of effectiveness and efficiency used to define usability (Argent et al. 2018; Brooke 1996). Moreover, most users claimed that the application gave them a sense of the natural world. The ITC-SOPI's ecological validity subscale result was not entirely satisfactory, which may be attributed to some of the questionnaire's questions that didn't apply to our application, like item B22, which was about how different smells made you feel in your environment. Also, the low degree of immersion offered by the tablet may help to explain this. The tablet, however, was chosen for this study because it can be used more quickly and easily than other tools for reproducing 360-degree media, and it could be simple to use at home, even without the guidance of a therapist or caregiver. Participants were at ease using the device because they were familiar with it and many of them claimed to do so frequently. These features achieve the goal of satisfaction, the third crucial aspect in defining usability (Argent et al. 2018; Brooke 1996). Participants were very satisfied with the application, and they expressed interest in using it. They were fascinated by using the tablet to explore the virtual environment and enjoyed performing exercises in this innovative way. Also, side effects and cybersickness were minimal.

On the other hand, the attractive experience that an immersive headset would offer could help to increase this application's acceptability. Future research will therefore compare task plays with the tablet and with a more immersive system, like cardboard or headset, to compare both experiences.

Despite promising results, some technical and interface problems were brought up by the TAP (Lewis 1982) during the usability study, and it is possible that these issues could be resolved by making the instructions clearer, adding a specific training phase, and improving the quality of images and videos. Thanks to these upgrades, the exercises should be easier to understand.

In the past, different studies developed various tools to assess memory successfully (Bruni et al. 2022a, b; Pieri et al. 2021; Pedroli et al. 2022), but there are focused on a limited aspect. This application provides an assessment of several memory components (i.e., visual memory and spatial memory) with the chance to include in the future even more tasks. In fact, according to our usability results, it will be possible to develop several other applications based on this technology for the assessment of other cognitive domains. Moreover, the application could be implemented, providing automatic data scoring and the possibility of sending it to the clinician. Additionally, future studies are needed to validate ECO-MEMORY by verifying its statistical properties. By comparing this instrument's results to those obtained from traditional memory tests (convergent validity) and other assessment tools that assess various domains (divergent validity), it may be possible to assess this instrument's capacity to evaluate memory domains. A further step might involve administering ECO-MEMORY to a major sample of subjects in order to produce normative data and take its potential application in clinical practice into consideration.

6 Limitations and conclusions

The present work is not exempt from limitations. Nowadays, the market of 360-degree devices offers considerably higher-quality omnidirectional cameras, which can give the measurements acquired a larger ecological value and higher image quality. This is the first technological gap. Another important consideration regards the low degree of immersion that constitutes one of the most important limitations of 360-degree media deployed with tablets, because of the lack of active navigation and the limited possibility of interaction within the environments. This could be a disadvantage in terms of engagement and feelings of naturalness in the presented scenario. At the same time, the lack of these features could have been experienced as an advantage instead of a limit for old people who did not have the proper skills to deal with more advanced technologies.

Our sample size requires a crucial consideration. We decided to use a limited number of users based on previous literature that used a restricted number of subjects to test usability (Guillén-Climent et al. 2021) as highlighted by Virzi and colleagues (Virzi 1992). The researchers state that most problems related to usability are detected with four or five subjects, who are less and less likely to reveal new information. We decided to select 13 users to cover a variety of demographic characteristics, technical skills, and levels of familiarity with the technologies; however, most of them reported the same usability problem at the TAP. Even with its limitations, these findings support the usability of 360-degree assessment aiding to limit the possible bias due to the personal characteristics of the trainer, implementing the objective evaluation of ecological situations.