Introduction

Worldwide, life expectancy is rapidly increasing and according to the previsions, the proportion of people older than 60 years will reach 21.1% by 2050 compared to the 9.2% in 1990 and the 11.7% in 2013 (World Health Organization; https://www.who.int/news-room/fact-sheets/detail/ageing-and-health). Despite the improvement of instruments and standards of care, late life is not empty of complex chronic medical conditions that represent major problems in particular for health care systems still focusing on a disease-oriented approach [1].

Accordingly, the transition towards an aging world is boosting a gradual passage to more person-focused care models [2]. Within these models, frailty, comorbidity, and multimorbidity have recently caught the attention of scholars and clinicians, with a significant increase of publications [3], even about the application of technological tools for diagnosis and intervention of these conditions. However, recent literature reviews underlined limitations linked to methodological procedures of research conducted in this field, reducing studies’ validity and results’ generalizability [4, 5]. Indeed, most of the studies testing technology feasibility use observational designs with assessment purposes in limited and heterogeneous samples of frail, comorbid, or multimorbid older patients, while interventional studies involving groups of patients defined by clear inclusion and exclusion criteria are under-represented. These methodological difficulties may be also linked to the interchangeable modality by which frailty, comorbidity, and multimorbidity are often defined, with a lack of consensus regarding their operational translation in practice [6]. Although frequently used as synonyms, these conditions are separate clinical concepts [7] that can independently coexist or mutually interact constituting pre-disability conditions [8].

There is an agreement depicting frailty as a dynamic and multicomponential condition depending on or leading to an extreme vulnerability to stressors and reduced resiliency [9, 10]. Despite this consensus, frailty has been alternatively defined as the result of the accumulation of deficits [11] or as a clinical syndrome (i.e., Frailty Phenotype) [7], with both these definitions failing to include symptoms belonging to psychosocial and cognitive domains, that are instead captured by the more recent concept of intrinsic capacity [12, 13].

The controversy about definitions of multimorbidity and comorbidity depends on the nature, time of onset, and assessment of the diagnosed clinical diseases [14] that co-occur, not interdependently, in the case of multimorbidity or that generate combined effects in reference to an index chronic disease in the case of comorbidity [15].

The prevalence of frailty, comorbidity and multimorbidity increases with aging [16], leading to reduce life expectancy, impairments in daily living activities and postoperative complications, increased risk of mortality, and costs for public health, in terms of frequency and duration of emergency room visits and hospital admissions [17,18,19,20]. Therefore, care pathways aiming at personalizing interventions based on the needs of older patients, namely patient-centered health systems, may be crucial to cover the urgency to manage the impactful long-term consequences of these conditions [17]. Going further, these novel care approaches could benefit from the use of assistive health technology (i.e., technological solutions aiming to maintain or improve functionality, autonomy, and well-being) or medical devices (i.e., technological solutions aiming to support prevention, diagnosis, and treatment) to prioritize integration between different settings of care and care professionals, the inclusion of caregivers in the treatment programs, development of remote self-management solutions and procedures [21,22,23].

Accordingly, in the last years, research showed that Information and Communication Technologies (ICTs), machine learning algorithms/models [24], and virtual reality tools [25, 26] may be beneficial for older people [27, 28]. In particular, technologies demonstrated to ameliorate diagnosis, prognosis, and prevention strategies [29] as well as facilitate remote monitoring [30], continuity of care [31], access to healthcare services [32] and even patients’ independence and quality of life [33].

However, studies pointed out also barriers to the use of technology in samples of frail, comorbid, and multimorbid older people, preventing cost-effectiveness analysis and solid integration of technologies within complex assistive models. Beyond technical and economic aspects [33, 34], as already mentioned, research seems to lack rigorous methodological approaches, with a clear displacement towards assessment of frailty, comorbidity, and multimorbidity compared to clinical trials testing technologies as intervention tools in samples suffering from complex geriatric syndromes. Indeed, technologies have been mostly tested in reference to their technical aspects, whereas applications in clinical settings frequently are pilot experiences on small and mixed groups of patients, not providing analysis of patients’ needs, considerations about usability and acceptability of tested devices, exploration of characteristics of the real-world application scenarios [4, 5].

Despite these limitations, to the best of our knowledge, literature still lacks a comprehensive review of research designs, in terms of observational or interventional designs, that studies using technologies for the assessment and treatment of aging-related syndromes in older people have applied so far.

This review aims to fill this gap by systematically describing research designs and procedures currently applied to transfer laboratory results to real-world practices in order to critically appraise studies’ methodological quality based on structured criteria and present considerations on how to standardize studies’ methods and research quality in this field.

Methods

Literature search

Our systematic review was conducted according to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines [35]. PROSPERO registration number: CRD42020218053.

The selected keywords were: “frailty”, “multimorbidity”, “comorbidity”, “aging”/“elderly” that were combined, into three arms, with ICT, machine learning, and virtual reality [33, 36,37,38,39]. See Appendix A in the Supplementary material for the details of the search strategies and combinations.

The arms were searched as major topics in Pubmed, Web of Science, and Embase (Ovid), restricting the literature search to title, abstract, and keywords.

Using a web and mobile systematic review manager [40], after duplicates removal, four blinded researchers (A.G., P.D.T., C.T., S.C.) in pairs categorized the records as “included”, “excluded”, or “unsure” based on title/abstract. Then, during the full-text screening, records included or categorized as “unsure” in the first stage were reviewed. In both screening stages conflicts were resolved by consensus of the researchers of each pair and a third author was consulted if discrepancies remained. The authors of papers whose full-text was not available were contacted.

Selection criteria

Eligible for inclusion were peer-reviewed studies published to the end of September 2020. The following hierarchy of eligibility criteria was adopted: (a) English written articles; (b) articles involving human samples; (c) peer-reviewed articles; (d) articles including frail or multimorbid or comorbid participants older than 65 years. In particular, we considered records in which frailty or multimorbidity or comorbidity were the main outcomes of the technology-based assessment or intervention, while we excluded studies where these conditions were simply assessed to describe participants but were not part of the studies’ aims; (e) articles applying technologies with clear diagnostic or intervention purposes. Therefore, we excluded records including technologies only to assess their usability or acceptability. Moreover, we included papers on telerobotics devices, while robotics used in surgical settings were excluded; (f) original peer-reviewed articles with interventional or observational study designs according to types proposed in [41], while narrative or systematic reviews, meta-analyses, case-reports, abstracts, conference proceedings and study protocols were excluded.

Data extraction and synthesis

Data from each of the included studies were collected by one of the authors in each pair by using a specific form. Data were checked for accuracy and completeness by the other pair’s member and discrepancies were solved by consensus and/or by a third author if needed (see Table 1).

Table 1 Characteristics of the included studies
Full size table

Quality assessment

We chose the most appropriate quality assessment tool based on the records’ study design defined according to [41]. In particular, interventional randomized and non-randomized clinical trials were assessed using the Cochrane Collaboration’s Risk-of-Bias Tool [42]. Observational studies with diagnostic aims were assessed using the Quality Assessment of Diagnostic Accuracy Studies tool – second version (QUADAS-2) [43]. Observational studies using predictive models with prognostic purposes were assessed through the Quality In Prognosis Studies (QUIPS) tool [44]. See Supplementary materials for details of the scales used.

Six blinded researchers (A.G., P.D.T., C.T., S.C., A.GR., F.R.G.) in pairs evaluated the studies’ quality. Conflicts were solved by consensus of authors in each pair or by the involvement of a third author in case of discrepancies.

Results

Review selection

Based on our keywords we retrieved 2207 records. After removing duplicates, we screened the title and abstract of 1626 papers. According to our selection criteria and following the consensus on conflicts, 290 studies moved then to the full-text screening. Among the 290 full texts, 34 papers met our inclusion criteria and were included in the qualitative synthesis (Fig. 1).

Fig. 1
figure 1

PRISMA Flowchart of the screening process

Full size image

Quality assessment

The quality and the risk of bias assessment of the retrieved studies are summarized in Figs. 2, 3 and 4. Considering the Cochrane Collaboration’s Risk-of-Bias Tool (Fig. 2), we evaluated 10 papers out of 34. Among these, six were randomized control trials [45,46,47,48,49,50], two were non-randomized control trials [51, 52], one was a pre-post design [53] and one was a cross-over randomized control trial [54]. The analysis of the risk of bias across studies revealed a high average quality with only two studies exposed to selection bias [52, 53] and two studies exposed to performance bias [45, 49].

Fig. 2
figure 2

Risk of bias evaluation with Cochrane tool

Full size image
Fig. 3a
figure 3

(a) Risk of bias evaluation with QUADAS-2 Risk tool. (b) (b)

Full size image
Fig. 4
figure 4

Risk of bias evaluation with QUIPS tool

Full size image

Based on the QUADAS-2 ratings (Fig. 3a), we assessed 15 [55,56,57,58,59,60,61,62,63,64,65,66,67,68,69] diagnostic accuracy studies, one retrospective cohort study [70], and one cross-sectional study [71]. The average quality of the included studies was intermediate to low as most of the analyzed items, in particular the tested index, the standard of reference, and timing of measurements, despite not totally at high risk of bias, were evaluated as unclear due to the lack of information in most of the retrieved papers. The item regarding the selection of patients was assessed as low risk by most of the papers, however, compared to the other items, it obtained the higher number of high risk ratings. Considering the evaluation of the applicability through QUADAS-2 (Fig. 3b), the quality of the retrieved observational studies was high.

We used the QUIPS tool to evaluate the quality of six observational studies describing a predictive model (Fig. 4). Among these, five papers were retrospective cohort studies [72,73,74,75,76] and one paper was a cross-sectional study [77]. The average quality was intermediate to low as the risk of attrition and confounding factors, as well as bias in patients’ section, were rated from moderate to high by most of the studies in this group.

Research methodology

The majority of the selected papers (24/34) reported observational studies, whereas only ten out of thirty-four studies were interventional researches (Table 1). Below we report the main results organizing the selected studies according to these two major methodological approaches.

Observational studies

Twenty-four out of thirty-four papers were observational studies and involved 923,319 patients affected by multimorbidity [70, 75, 78], 90,096 affected by frailty [55,56,57,58,59,60,61,62,63,64,65,66,67,68,69, 72,73,74, 76] and 1,765 patients affected by geriatric syndromes [71, 77]. The diagnostic accuracy study design was the most represented study typology, with fifteen out of twenty-four studies [55,56,57,58,59,60,61,62,63,64,65,66,67,68,69], and the general aim of all these studies was to assess a diagnostic methodology devoted to detect frailty. The second most represented study typology was the retrospective cohort study design, with six out of twenty-four studies. The general objective of five of these studies was to elaborate a predictive model for frailty [72,73,74, 76] or multimorbidity [75], whereas the general aim of the last one was to study the mortality rate of patients suffering from multimorbidity [70]. The typology of the remaining three papers refered to cross-sectional study design. The general aim of two of these was to study prevalence of geriatric syndromes [71, 77] and the third study aimed to elaborate a predictive model for multimorbidity [78].

The twenty-four observational studies selected implemented different ICT technologies. Among the fifteen diagnostic accuracy studies, wearable sensors were the most common ICT technologies used and were mainly proposed to analyze postural and movement variables [55, 59,60,61, 67]. Four studies proposed to exploit widely diffuse technologies (smartphone or tablet) to analyze movement parameters during sit-to-stand tasks [64,65,66] or handwriting [63]. Two studies implemented virtual reality technologies, one to propose serious games [58] and the other to implement movement analysis [57]. The remaining four studies were based on different technologies: one study implemented a social robot to administer clinical questionnaires [62], another one assessed motion and walking parameters using commercial motion capture sensors like Kinect [68], a third one implemented a home monitoring platform with ambient sensors to analyze patient daily habits [69], and the last study used machine learning to identify frailty from administrative data [56]. Among the six retrospective cohort studies selected, five exploited different machine learning methods to develop predictive models for frailty [72,73,74, 76] or multimorbidity [75], whereas one study analyzed data from a telemonitoring experience to assess mortality according to multimorbidity and telemonitoring status [70]. The remaining three cross sectional studies implemented, in two cases, machine learning algorithms to generate predictive models for multimorbidity [78] or to detect patterns of medication combinations according to geriatric syndrome status [77], and in one case a mobile iPad application was used to study the prevalence of geriatric syndromes [71].

Interventional studies

Ten out thirty-four papers were interventional studies and involved 1912 patients affected by multimorbidity [46,47,48,49, 52], 170 affected by frailty [45, 50, 51, 54] and 53 with comorbidity [53].

Seven studies were randomized controlled trials, four devoted to study interventions for multimorbidity [46,47,48,49] and three for frailty [45, 50, 54]. The remaining three studies were two non-randomized trials, one for frailty [51] and one for multimorbidity [52], and one a pre-post study design which analyzes an intervention for comorbidity [53].

The seven randomized controlled trials selected used different ICT technologies. Three studies promoted physical activity (PA) or motivation toward PA through virtual reality [50], robot [54] or a web-based motivational program [49]; two studies analysed Ambient Assisted Living (AAL) interventions [45, 48], one study used telemonitoring [47] and another one a video consultation intervention [46]. The three further non-randomized trials proposed a smart-phone based PA program [51], a telemonitoring system [53] or an integrated care intervention [52].

Discussion

The present review first aimed to systematically describe research designs implemented by studies about technologies applications to clinical assessment and treatment of aging-related syndromes. Overall, the results showed a clear imbalance toward a more represented amount of observational studies compared to interventional ones. This result reflects the well-known limits of applying only standard Randomized Clinical Trials (RCTs) in the research field of technology-based interventions for rehabilitation purposes [79]. Moreover, the quality assessment revealed that interventional studies were of higher quality, whereas observational studies were mainly of intermediate to low quality. Taken together, these findings may suggest that the field is still seminal as emerged in a previous review [4].

Our second aim was to propose a step model to standardize studies’ methods and improve the research quality in this field. This is in line with the actions of the European Network for Health Technology Assessment. This network on technology research recommends: a clear assessment of previous studies’ results; the disclosure of the rationale for using technology; the clinical indication of the population, the kind of intervention and comparators; the evidence about safety and effectiveness; the definition of study design (see https://www.eunethta.eu/methodology-guidelines/). Based on these recommendations, we hypothesize that one possible interpretation of our results could be the absence of a strong frame of reference describing all the steps useful to obtain a technology of good quality to be used for diagnostic or interventional purposes in a real-world setting. Considering the field of new drugs development, suggestions on how to formulate a canonized frame helping to overcome the emerged limitations could be found. Indeed, we propose that the development of a new technology to be used for diagnostic or interventional purposes has to pass several steps similar to those of the process to develop a new drug, as shown in Table 2.

Table 2 5-step Model
Full size table

In our model, the two phases of the “Clinical research” step represent the attempt to exploit the power of “Efficacy” studies, which are intended to assess the performance of an intervention under ideal circumstances, as well as “Effectiveness” studies, which are intended to produce evidence of therapeutic effectiveness in real-world practice settings [80, 81]. Usually “Efficacy” studies are the classical RCTs that, even though represent the gold standard for evaluating the efficacy of an intervention, require highly controlled conditions to avoid biases and confounding factors [81]. On the other hand, “Effectiveness” studies are retrospective or prospective real-world observational studies that, by using a less strict methodology and examining interventions under circumstances closer to real-world practice, lead to complementary evidence to that provided by RCTs, even if they are more prone to several sources of bias and risk of uncontrolled confounders [80].

Considering these characteristics and the concerns of limiting the research on technologies for disabilities only to RCTs [79], we think that both these types of studies should contribute to the research field of technologies applications for the diagnosis and intervention of age-related pathologies. Indeed, the results of this review, coherently with previous works [4, 5], show that studies in which technological solutions are tested in samples of frail, comorbid, and multimorbid old patients frequently describe technical aspects of technologies through laboratory experimentations, with only empirical applications on pilot samples. In other cases, they are large but methodologically poor observational studies aiming at improving the assessment or at providing predictive models. Sometimes, they are clinical trials that, despite the good quality, lack usability and acceptability considerations and are underpowered to generalize the results or to run cost-effectiveness analyses. Therefore, we first suggest that systematizing “Efficacy” and “Effectiveness” study designs in the “Clinical research” phase of our model could enhance the methodological rigor of randomized trials and observational studies, both conducted on small selected samples as well as on heterogeneous and large groups of patients. Following the methodological requirements of “Efficacy” and “Effectiveness” research designs, RCTs could indeed rightly test technologies efficacy, while observational studies could strictly explore aging-related syndromes prevalence, technologies applications’ validity, reliability, sensitivity, and their role in predicting long-term outcomes of chronic conditions. Second, considering the proposed model as a whole, we are confident that applying such a rigorous framework could help scholars to dialogue with clinicians, to effectively investigate technologies’ usability, acceptability, and safety based on clinical population characteristics and strata. This, in turn, could prepare the stage for well-designed clinical studies that could provide solid results, even regarding cost-effectiveness analysis, to be used for revision and approval by a recognized Institution and finally for post-market monitoring of long-term effects and large-scale use.

In conclusion, the results of the present systematic review seem to suggest that research in the field of the development and use of technological tools for aging-related syndromes is, at the moment, mostly oriented toward observational studies devoted to diagnostic tools to be used in the assessment of geriatric conditions (e.g. frailty). The field suffers from some limitations related to the research quality and poor attention to interventional studies of efficacy and effectiveness. We propose that a structured and shared methodological approach, like that followed in pharmaceutical studies, could help the field to increase the research quality and more adequately respond to the needs of patients and their caregivers.