Sensor, wearable, and remote patient monitoring technologies are typically used in conjunction with video and/or in-person care for a variety of interventions and care outcomes. This scoping review identifies clinical skills (i.e., competencies) needed to ensure quality care and approaches for organizations to implement and evaluate these technologies. The literature search focused on four concept areas: (1) competencies; (2) sensors, wearables, and remote patient monitoring; (3) mobile, asynchronous, and synchronous technologies; and (4) behavioral health. From 2846 potential references, two authors assessed abstracts for 2828 and, full text for 521, with 111 papers directly relevant to the concept areas. These new technologies integrate health, lifestyle, and clinical care, and they contextually change the culture of care and training—with more time for engagement, continuity of experience, and dynamic data for decision-making for both patients and clinicians. This poses challenges for users (e.g., keeping up, education/training, skills) and healthcare organizations. Based on the clinical studies and informed by clinical informatics, video, social media, and mobile health, a framework of competencies is proposed with three learner levels (novice/advanced beginner, competent/proficient, advanced/expert). Examples are provided to apply the competencies to care, and suggestions are offered on curricular methodologies, faculty development, and institutional practices (e-culture, professionalism, change). Some academic health centers and health systems may naturally assume that clinicians and systems are adapting, but clinical, technological, and administrative workflow—much less skill development—lags. Competencies need to be discrete, measurable, implemented, and evaluated to ensure the quality of care and integrate missions.
The number of connected wearable devices worldwide is expected to increase from 325 million in 2016 to 929 million by 2021, with 1 in 6 Americans using a wearable by 2021 (Piwek et al. 2016; Statistica 2018). Sensor, wearable, and remote patient monitoring (RPM) technologies are not only being integrated into care but are also increasingly used as the primary assessment and/or treatment modality (Hermens and Vollenbroek-Hutton 2008; Kvedar et al. 2014). Examples of their use include the collection of behavioral data, monitoring symptoms and adherence, and providing preventive and therapeutic interventions (Hilty et al. 2020; Luxton 2016; Silva et al. 2015; Steinhubl et al. 2013), making care more person- and patient-centered.
Sensor, wearable, and RPM technologies can transform care by moving from cross-sectional, manual transfer of data at a healthcare appointment to a 24 × 7, longitudinal framework (Areàn et al. 2016; Luxton 2016; Torous and Roberts 2017; Ariga et al. 2019). Enabling technologies, including cloud computing, ambient sensors, and artificial intelligence (AI) (e.g., machine learning (ML)), make the collection, processing, and sharing of data far more integrated and provides the opportunity for real-time feedback based on the ecology (home, health, lifestyle, social) of patients in natural settings (Seko et al. 2014). These data support “in-time” clinical decision support (Rohani et al. 2018; CDS) (Greenes et al. 2018) and automatic monitoring systems (Garcia-Ceja et al. 2018; National Academy of Sciences, Engineering and Medicine 2020). For example, digital phenotyping or behavioral markers can correlate multimodal sensor data, cognitions and depressive mood symptoms (Mohr et al. 2017) to detect illness and/or predict relapses.
The COVID-19 pandemic forced healthcare providers and systems to rapidly adapt their way of providing services. This also necessitated the need for rapidly attainment of technological capabilities and practical skills to assure competent, legal, and ethical practice. Fortunately, competencies are identified and training and knowledge resources for video-based telehealth services were well established and are generally accessible. However, training and competency resources for wearables, sensors, and remote monitoring technologies are presently limited, and the existing literature mostly describe methods, interventions, technologies, and care outcomes, rather than required clinical skills to use technologies (Hilty et al. 2020). Even clinicians familiar with telehealth may not see that these technologies are part of care, struggle to use them, or fail to realize their limitations (Hilty et al. 2018; Hilty et al. 2019a, b; Hilty et al. 2020). Graduate education, graduate education councils, and professional organizations/boards are providing guidance on use of technology but have not yet put forward competencies to ensure quality care for sensor, wearable, and RPM (Hilty et al. 2017; Maheu et al. 2019).
Given the increasing capabilities and benefits of sensors, wearables, and RPM and the current lack of identified competencies regarding use in clinical care, we aim to identify essential clinical skills and organize them into competencies within an educational theory, implementation science, and evaluation practices framework. We also consider that, like clinical interventions, competency implementation and effectiveness depend on assessment of acceptability, adoption, feasibility, costs, and sustainability (Proctor et al. 2010; Curran et al. 2012; Gargon et al. 2019; Marcolino et al. 2019). Identified and clarified competencies will help healthcare providers and systems adapt to and integrate these technologies into workflow and advance person—and patient-centered care.
This scoping review does not specifically address accreditation, licensing, and other regulatory boards/agencies but briefly summarizes position statements and guidelines on the integration of technology in clinical settings.
The original six-stage process (Arksey and O’Malley 2005) and updated modifications (Levac et al. 2010) for scoping reviews were used.
This review addresses the overarching question: “What are the skills, attitudes, and knowledge that trainees and clinicians need to incorporate sensors, wearables and RPM into clinical practice?” It synthesizes clinical data based on research findings and considers current and target (i.e., future/ideal) states for those using sensors and wearable technologies.
Identifying Relevant Studies
The literature key word search was conducted for articles published through May 2020 using PubMed/Medline, American Psychological Association (APA) PsycNET, Cochrane, Embase, PsycINFO, Web of Science and Scopus, Science Citation Index, Social Sciences Citation Index, Telemedicine Information Exchange database, Centre for Reviews and Dissemination, and The Cochrane Library Controlled Trial Registry database. The initial search targeted four concept areas and associated search terms:
Competencies (adult, assessment, attitude, behavior, cognition, cognitive, curriculum(a), education(al), evaluation, framework, goals, graduate, improvement, knowledge, learner, learning, literacy, objectives, pedagogy, postgraduate, process, quality, resident, skill, student, supervision, teaching, trainee, training, milestones)
Sensor, wearable, and remote monitoring (actigraphy, activity, assessment, data, diagnosis, ecological, geographical, global, GPS, momentary, monitoring, passive, remote, storage, telemonitoring)
Mobile (android, app, asynchronous, cellular, device, e, ehealth, ebehavioral, e-behavioral, e-mental, health, mhealth, m-health, mobile, phone, smartphone, tablet, text), asynchronous (app, device, eConsult, e-consult, e-mail, mobile, monitoring, sensor, social media, store-and-forward, technology, text, wearable) and synchronous telepsychiatry, telebehavioral or telemental health (video, web-based, Internet)
Behavioral health (BH), psychiatry, and psychology (behavioral, client, clinician, care, diagnosis, emotional, health, intervention, medicine, mental, patient, services, psychiatry(ic), psychology(ical), therapist, treatment)
Two authors (DH, CA) searched then screened titles and abstracts for relevant studies. A third reviewer (EK) determined outcomes if disagreements occurred. Full-text articles were reviewed for final inclusion based on the key word search. Reviewers met at the beginning, midpoint, and final stages of review to discuss challenges and uncertainties and refine the search strategy. The flow diagram is in Fig. 1. Inclusion criteria were the terms in combination from the four concept areas: 1 and 2; 1, 2, and 4; or 1, 3, and 4. Exclusion criteria included the following: terms out of context (e.g., abstract use of competency, not mentioning a single skill, non-measurable behavior); data unrelated to BH; not applicable to clinical care; wrong report type (e.g., abstract, editorial, column, literature review); models, principles, and concepts (e.g., cognitive, milestones, patient) with no educational outcomes; non-English language; and other.
A data charting form was used to extract data, and notes were organized consistent with a narrative review or descriptive analytical method by each reviewer. The reviewers compared and consolidated information using a qualitative content analysis approach. The information was shared with selected experts (see below), their input summarized, and themes extracted.
Analysis, Reporting, and the Meaning of Findings
This phase organized results in a table, with skills outlined and parsed. There were so few papers that findings were reported individually since the depth of existing research was less than expected. Therefore, the reporting used excerpts from published framework tables. The Accreditation Council of Graduate Medical Education (ACGME) Milestone framework was selected as it is well-known across medicine (ACGME 2013). Its six domains are patient care, medical knowledge, system-based practice, professionalism, practice-based learning and improvement, and interpersonal and communication skills. Two authors (DH, CA) examined the ACGME milestones from addiction medicine, clinical informatics, family medicine, internal medicine, psychiatry, and surgery, the CanMEDs roles for physicians (Royal College of Physicians and Surgeons Canada 2015), and the Coalition of Technology in Behavioral Science (CTiBS) telebehavioral health competency framework (Maheu et al. 2018) to build the current competency framework (Hilty et al. 2018). The content was organized into a 3-level learner system (i.e., novice/advanced beginner, competent/proficient, and advanced/expert) (Dreyfus and Dreyfus 1980; ACGME 2013; Hilty et al. 2015).
Expert opinions were solicited to review preliminary findings and suggest additional steps for improvement. A list of relevant experts was compiled from (1) recent regional and national presentations (e.g., American Telemedicine Association (ATA), American Psychiatric Association, American Association of Directors of Psychiatry Residency Training), (2) technology-related special interest groups of organizations, (3) national behavioral health organizations (e.g., CTiBS), and (4) authors of recent articles. The goal was to have representatives with mobile health and informatics expertise in behavioral analysis, counseling, couples/marriage/family, psychiatry/medicine, psychology, social work, and substance use professions.
Fifty-two experts were invited from the Department of Veterans Affairs, Department of Defense, and academic institutions across four countries (Canada, India, Italy, USA). Disciplines included 29 (55.8%) psychiatrists, 16 (30.8%) psychologists, 3 (5.8%) engineers, 1 (1.9%) social worker, 1 (1.9%) occupational therapist, 1 (1.9%) dietitian, and 1 (1.9%) primary care physician.
Of the 52 experts invited, 26 responded and 24 (46.1%) participated, with 11 (45.8%) via email, 9 (37.5%) via 1 of 4 videoconference sessions, and 4 (16.7%) used both options. Feedback was collated based on previous work using consensus and modified Delphi processes (Hilty et al. 2018, Maheu et al. 2018). The 24 experts were asked to complete a brief 6-item Likert-scale survey and 15 (62.5%) responded. Results showed that the majority agreed or strongly agreed that:
“Table 1 was helpful in understanding the scope of what is currently being used” (100%).
“The competencies provided in Table 3 are in the ballpark and relatively ‘complete’” (85.7%).
3 “The length of Table 3 is good (not too long, not too short)” (78.6%).
“The subdomains in Table 3 are helpful” (92.9%).
“The behaviors/skills in Table 3 are learnable, teachable and measurable” (100%).
“The level of clarity and detail in Table 3 is adequate (not too much, not too little)” (87.6%).
The abstract, tables, and manuscript were edited based on the feedback.
Figure 1 shows that from a total of 2846 potential references, two authors (DH, CA) found 2828 eligible for title/abstract review, and 521 for full text review. The majority of studies were excluded due to terms used out of review context and reviewing references revealed more papers. The final number of papers included in the review was 111 (highlighted in grey in References).
Sensor, Wearable, and Remote Patient Monitoring Technologies Evidence-Based
A variety of technologies were used for mood and anxiety disorders (Hilty et al. 2020), with smartphones (66.3%), wristbands or smartwatches (22.8%), and holters (6.5%) the most common (Table 1). The most common sensors (e.g., smartwatches, heart rate monitors, smart glasses; L’Hommedieu et al. 2019) were accelerometers (50.0%), phones (39.1%), GPS (35.9%), microphones (30.4%), actigraphs (25.0%), and electrocardiograms (ECG) (25.0%), and implantable medical devices are used for other conditions (Ferguson and Redish 2011). The sensors convert human biological data into digital signals for a microprocessor to interpret, and then, the flow of information continues (Heikenfeld Jajack et al. 2018).
Most studies used instruments relevant to their application domain, including questionnaires, ecologic momentary assessment (EMA), and clinician assessments. These are objective, accurate (although not always validated as such), and can validate self-report data (e.g., Fitbit). There was significant variation in the timing of validation measures, from one-time measures to many per day or pre-post assessments. For data processing, software applications that communicated with a remote server were used in most studies to save data for processing and, at times, provide within-study feedback to participants.
Remote monitoring and remotely supervised treatment or training (RMT) uses complex distributed information and communication technology (ICT) systems to measure, transport, and store information (ATA 2014). Feedback helps clinicians with clinical decision support and to increase patient awareness of their health status, gives them a feeling of safety and motivate, and enables them to change/improve their health status (Maddison Cartledge et al. 2019).
One major subset of studies focused on feedback in the form of applications that display graphs representing mental health status (Abdullah et al. 2016; Faurholt-Jepsen et al. 2016). Many studies found that motivational messages (e.g., specific suggestion for action), tailored messages (e.g., specific to disease, stage of progress, demographic information, treatment plan), and health professional contact-enhanced therapeutic engagement are generally more effective than receiving (typically numeric) health measure feedback (Ma et al. 2014). These individualized (or similar user) models appear more effective (Wahle et al. 2016) than generalized models (e.g., generic message for a depression cohort). Individualized design of training and exercise programs may be helpful to maximize the improvements in performance and avoid risks (Cardinale and Varley 2017).
Another area of research is related to validating diagnosis and/or determining predictors of outcomes. Within the depression studies, cell tower identification (i.e., geographic movement), screen active duration, and activity measurement have the strongest linkage with depression severity compared with distance travelled and SMS text message received, whereas manic patients have longer calls, more messages sent, and more travel (Hilty et al. 2020). For bipolar disorder, studies generally support the interpretation of sensed data by employing traditional measures or assessments for depression (e.g., PHQ-8 or PHQ-9 self-report instruments) and clinician assessment/batteries (Faurholt-Jepsen et al. 2016). Mood and anxiety studies use algorithms to interpret or predict participant’s status (e.g., support vector machine, naïve Bayes classifiers, decision trees, random forests, linear regression, Bayesian networks, logistic regression, and other machine learning methods) (Elgendi and Menon 2019). The College Sleep Project used multi-modal sensors (physiological, behavioral, environmental, social) from smartphones to identify associations among and develop predictors for sleep behaviors, stress, mental health, and academic performance (Sano et al. 2015).
Behavioral Health Profession and International Organization Position Statements and Guidelines on Telehealth and E-health
Our current review builds on past comparisons (Hilty et al. 2017; Hilty et al. 2018; Hilty et al. 2020) of health and behavioral health professional organizations’ documents that pertain to technologies by focusing on sensors and wearables (Table 2). US and Canadian guidelines were reviewed (American Psychiatric Association 2014, 2017, 2018; American Psychiatric Association-ATA 2018; ATA 2009, 2013, 2017; Royal College of Physicians and Surgeons Canada 2015). Only the ATA child and adolescent guideline mentions monitoring and wearables (ATA 2017). The only competencies have come from individual research groups for video (Hilty et al. 2015, 2018), social media (Hilty et al. 2018; Zalpuri et al. 2018), mobile health (Hilty et al. 2019a, b; Hilty et al. 2020), and asynchronous (Hilty et al. 2019a, b; Hilty et al. 2020) technologies.
Contributions from behavioral health come from American, Australian, British, and Canadian psychology associations (APA 2013, 2019; Association of State and Provincial Psychology Boards 2014; Australian Psychological Society 2011; British Association for Counselling and Psychotherapy 2019; Canadian Psychological Association 2006; Goss et al. 2001) and U.S. social work (National Association of Social Workers 2017), counseling (American Counseling Association 2014; Canadian Counselling and Psychotherapy Association 2019), and couples/marriage family therapy (American Association for Marriage and Family Therapy 2015; American Association for Marriage and Family Therapy Regulatory Board 2016). The most commonly discussed technologies were e-mail and social media, followed by texting, with note of sensors and wearables by the Canadians (sensor, text, wearable) and Australians (device, e-consultation, mobile, text). Only the CTiBS competencies for telebehavioral health (Maheu et al. 2018) mentioned briefly sensors and wearables as part of a mobile health domain.
Institutional and professional organizations are overseen by accreditation, licensing and boards/agencies to provide guidance to ensure professionals provide care proficiently, legally and ethically. They proactively inform and collaborate with regulatory boards/agencies to address broad, overarching issues related to quality of care, standards for practice and maintenance of certification. They also employ competency benchmarks (APA 2012), skills-based exams, and decision-making exams (Hilty et al. 2020). They tend to make changes incrementally and adjust to sentinel events one-by-one.
Based on the current review, the landscape related to sensors and wearables reveals several themes regarding mobile health technologies, some of which are concerning with respect to competencies (Seppälä et al. 2019; Hilty et al. 2020). First, there is incremental adaptation of technology added to care systems with the assumption that it is just part of routine delivery rather than a new form of care (Hilty et al. 2013). Second, innovation is siloed within research programs and/or pilot projects. Third, most innovation is related to video, e-mail, and electronic health record (EHR) portals, not mobile health, sensors, and wearables. Fourth, few if any academic health centers have yet to create or adopt technology competencies or install mobile health (apps, text mobile phones) and wearable sensors into clinical decision support systems (CDSS), so impact is limited.
Shifting Perspectives for Patients, Clinicians, Teams, and Systems
The future state is an “open” mobile health architecture that offers universal data standards and a global interconnected network, with patient and clinician facing dashboards, rather than apps and other technologies limited by siloed or un-integrated proprietary data, management, and analytic tools (Ramkumar et al. 2019). Clinicians may have to reconsider patient’s skills, needs, and preferences to determine suitability for any specific technology as part of the consent process, and again later, when needs/preferences among available choices are understood or care plans change (Hilty et al. 2015; Maheu et al. 2018).
The impact of new technology has increased exponentially (Mermelstein et al. 2017; Sturmberg et al. 2014), with users currently compared as digital natives versus immigrants (Wang et al. 2013). A broader perspective from education, business, health care, and other disciplines suggests that technology plays different roles along an augmentation-exploration-innovation spectrum (Hilty et al. 2019). It may affirm, complement, accentuate, and augment what people do. Further, it may help people explore, engage, and experiment with ideas and partners in new ways, “changing” dynamics or facilitating a different culture, making new things possible.
Paradigm Shifts in Integrating Sensors and Wearables with In-person and Synchronous Care
The first paradigm shift is including technology in traditional health care or adapting that care to technology. This can be difficult for clinicians, training programs, and health systems. The first example of this shift was from in-person to synchronous and asynchronous video (Yellowlees et al. 2018; Hilty et al. 2020). Even low-cost systems can facilitate engagement and “social presence” for participants to share a virtual space, get to know one another, and discuss complex issues (Hilty, Nesbitt et al. 2002). The most important goal with technology is to keep the exchange of information, clarity, responsiveness, and comfort close to real-time experiences (Hilty et al. 2020). Empathy, positive regard, and genuineness play a critical role in therapy effectiveness (Grekin et al. 2019; Kemp et al. 2020). Clinicians must adjust techniques to inquire, engage, communicate, and listen effectively (Hilty et al. 2020).
A second shift is correspondence via technology outside the office visit creating a new sense of continuity, connection, and exchange of information. Patients, clinicians, and healthcare researchers are trying to understand how this affects the therapeutic frame, communication, boundaries, and trust (Hilty et al. 2020; Li et al. 2016). A primary example of this is social media, which became a key part of patients’ lives, and by default part of care, though many clinicians did not understand or embrace it (Hilty et al. 2018). Asynchronous communications have a sense of immediacy and interaction that build “trust” but, despite a sense of control, are not always clear and can lead to ill-intended consequences (Hilty et al. 2020). Administrative planning and training can manage risks (e.g., privacy, self-disclosure, cyberbullying) (Armontrout et al. 2016; Joshi et al. 2019).
A third shift is how to integrate mobile health, text, sensors, social media, and other technologies into clinical workflow for a virtual office. As apps became popular, clinicians adapted more quickly, and now clinicians need to assess sensors and wearables for potential applications (e.g., entertainment, health care, behavioral health) (Hilty et al. 2019b). This is a key part of the therapeutic relationship, and there is a flow to the process (Fig. 2). For example, in-time data collection can alert a patient to self-assess mood regularly, with data transmitted automatically to the clinician. In-time data collection (e.g., EMA) is more valid, reliable, and meaningful than sporadic, retrospective reflection/recall, as long as the time to input data does not adversely impact patient motivation (Russell and Gajos 2020). This process also facilitates in-time clinical decision-making by adding a cognitive scaffolding and analytic methods that enhance confidence and skills (Clark 2013; Luxton 2016) (e.g., mood and medication management) (Thompson et al. 2014). This requires flexibility, between visit review of data, and supervision for trainees.
A fourth shift relates to organizational leadership and change. Many organizations are reluctant to change regarding technology despite telehealth’s impact (Hilty et al. 2020). Progressive businesses integrate core divisions (e.g., research, operations, finance) with information technology (IT) (i.e., shared IT-business framework) to leverage knowledge and capital (Ray et al. 2007). This paradigm has been applied to telehealth to organize/integrate health care rather than adding technology to existing systems (Hilty et al. 2019), suggesting that organizing care with technology will have better outcomes than appending it to health care.
Impact on Clinical, Technological, and Administrative Workflows
For healthcare systems, these new technologies have many implications, including clinical, technical, policy, and other administrative workflows to comply with legal and ethical mandates (American Psychiatric Association-ATA 2018; ATA 2009, 2013, 2017). Technological factors relate to access (i.e., equipment, software) and ease of use, as well as participant literacy and skill. Some tools are only new versions of in-person tools (e.g., smartwatch to collect photoplethysmography (PPG) signals to capture heart rate variability, an index associated with overall emotional and physical well-being), while other technologies shift perspectives dramatically for trainees and clinicians, requiring new competencies. For example, speech recognition and natural language processing (NLP) simplify data processing with a more natural and transparent interaction experience (e.g., Siri, Watson) (Witt et al. 2019) but require new skills.
The PACMAD (People at the Centre of Mobile Application Development) usability model emphasizes seven factors: effectiveness, efficacy, satisfaction, learnability, memorability, errors, and cognitive load (Harrison et al. 2013). A structured approach is needed to fit a sensor, app, or other technology to a user and evaluate its success, impact and value (Caulfield et al. 2019). Steps could include requirement definition (e.g., unobtrusive, body location, degree of measurement accuracy), device search, device evaluation criteria (e.g., human factor measurement tested, regulatory compliance, technical specifications/capabilities), and post-implementation evaluation of fit (or alternatively, use in others who are very similar to a give patient). While most clinicians currently lack these skills, the increased need to offer RPM and telehealth options during the COVID-19 pandemic has now increased utilization, and training should be based on established competencies when possible.
Many fields use human-centered co-design studios to develop and pre-test technological options (Hetrick et al. 2018; Russell et al. 2020). Focus groups, studies, and surveys have suggested ways to improve technology in health care through involvement and supervision of caregiver(s), better visuals, more immediate chat options and feedback between patients and clinicians, gamification and incentives, reminder alarms/alerts (MacKintosh et al. 2019), and seamless integration of data from apps and mobile phones into patient portals (Bennett et al. 2012).
Remote monitoring helps clinicians and trainees see or experience how important technology is for patients (e.g., safety, empowerment, control, convenience), healthcare systems (outpatient/in home, stepped care), and payers (e.g., insurance, cost reduction) (Hermens et al. 2008; Ramkumar et al. 2019). More importantly, they see how interprofessional teams can be effective, efficient, and collaborative in sharing expertise and roles (e.g., care coordinators, nurses) (Barisone et al. 2018; Winters et al. 2018).
Ethical issues related to informed consent, privacy, and data security are important to consider. Ethical issues concern providing and obtaining adequate informed consent from participants, information quality in monitoring apps, personal privacy, and international legal and cultural differences (Luxton 2016; Torous and Roberts 2017). For example, it is possible that with a combination of a great deal of physiological data, a person could theoretically be identifiable. Mobile health may also create privacy issues associated with third parties (e.g., images and sounds from nonparticipants). All of these issues should be evaluated during the technology design process so appropriate encryption and data security features are built in (Luxton et al. 2012). Other technological challenges include sensor precision, power, location, analytic procedure, communication, data acquisition, and processing, as well as performance issues between devices, accuracy of date versus gold standards, error rates, and other parameters (Patel et al. 2002).
Sensor and wearable technologies have not been adopted significantly to date, partly due to security, affordability, user-friendliness, compatibility with EHR systems (Bennett et al. 2012; Dinh-Le et al.2019), and financial and policy issues that have not been resolved even though virtual care codes were approved for some services in the U.S. in 2019 (Hilty et al. 2020). Manufacturers and healthcare providers also must consider certification and approval requirements related to US, European Union, and other international regulatory standards (Loncar-Turukalo et al. 2019; Ravizza De Maria et al. 2019).
Health System and Institutional Approaches to Sensor and Wearable Competencies
Competency-based education focuses on clinical skill development in addition to knowledge acquisition (Iobst et al. 2010). The Clinical Informatics Milestones have been suggested for the EHR, data use (e.g., safety, quality, privacy), and CDS (Hersh et al. 2014). These were followed by adult telepsychiatry (Hilty et al. 2015, 2018), CTiBS’ telebehavioral health (Maheu et al. 2018), social media (Hilty et al. 2018; Zalpuri et al. 2018), mobile health (Hilty et al. 2019a; Hilty et al. 2020), and asynchronous (Hilty et al. 2020) competency sets. They include suggestions for training, faculty development, and institutional change and use the ACGME domains (ACGME 2013) with novice/advanced beginner, competent/proficient, and expert levels. Sensors and wearables have yet to be addressed.
The slow integration of telehealth into clinical care and curricula may be a revealing story for sensors and wearables (Hilty et al. 2019). Interest by interns, residents, and fellows has significantly increased, but trainees across behavioral health have attributed inadequate experience, false beliefs of teachers, and speculation about technology efficacy as key limiting factors (Glover et al. 2013; Hilty et al. 2015). Residents have had valuable tele-experiences in establishing rapport and engaging with patients, working collaboratively, identifying different approaches used, and becoming aware of how to handle complex cases (Teshima et al. 2016). Many experts have called for more work in telehealth education (Balon et al. 2015; Sunderji et al. 2015; Crawford et al. 2016; Khan and Ramtekkar 2019), and one survey revealed only 21 of 46 programs have a telehealth curriculum or informal clinical experience (Hoffman and Kane 2015). The Veterans Affairs’ national telemental health training program provides a model, with comprehensive web-based modules, national satellite broadcasts, a national teleconferenced journal club, interactive Internet applications, and live real-time simulated remote competency assessments (Gifford et al. 2012; Godleski 2012).
Mobile health, social media, and informatics practices require more substantial planning and organizational change (Hilty et al. 2018; Hilty et al. 2019a, b, 2020; Hilty et al. 2020). Building on the IT-business model, this requires a multi-dimensional shift in training and clinical culture (Fig. 3). It requires positive attitudes in clinicians, teachers, and faculty to positively support learners (Mostaghimi et al. 2017) and reflect on practice change, share information, and adjust curriculum (e.g., courses, rotations, supervisory approaches). For sensors and wearables, health systems need to integrate objective data into clinical workflow (Dogan et al. 2017). EHRs need to develop comprehensive use-case scenarios that include details regarding the type of data needed to support care processes, clinical and community-based personnel, and standardized interventions that are already in place or need to be established to best utilize the information to be collected and integrated. These use-case scenarios should also be specific for different clinical populations (Lobelo et al. 2016).
Sensor and wearable curricular approaches are similar to those for mobile health and asynchronous competencies (Hilty, Chan et al. 2019a, b, 2020; Hilty et al. 2020). In addition to the sensors being the focus (i.e., content) of these competencies, sensors can shape the learning process, teaching learners, and simultaneously giving them experience (e.g., armband to measure proper hand washing via signal processing and ML) (Kutafina et al. 2015). Learning analytics can be applied to cluster, depict, and process data to support a number of objectives (González-Crespo and Burgos 2019), since teachers need to reshape the course plan for different learners (e.g., styles, motivation, performance) and integrate that information with real-time analytical information to supervise, assess, adapt, and offer feedback. Sensors and wearables can be applied to a wide range of educational contexts, including but not limited to:
learning management systems (LMS) or content management systems (CMS) that process user data collected
studying relationships between type of device from which users access LMS and how it affects performance
live or virtual reality (VR)-based sessions with teams or patients so oral and body language skills are shaped and improved
game-based learning with virtual/augmented reality to improve procedures
recommendations to users (e.g., explore impact of order given, behavioral response, impact of choices on engagement via algorithm) (Baldominos and Quintana 2019; Roque et al. 2019)
Many healthcare institutions already provide students with iPads, smartphones, and other technologies (Gaglani and Topol 2014), so adding sensors and wearables is the next logical step, allowing for enhanced guidance from preceptors on technique and interpretation of findings. Inter-institutional collaborations also improve teaching, reduce costs, and integrate a variety of patient populations (Metter et al. 2006).
Sensor, Wearable, and Remote Patient Monitoring Competencies
The proposed framework (Table 3; domains vertically, learner levels horizontally) uses the six ACGME domains (ACGME 2013). Most content came from the telepsychiatry (general themes) (Hilty et al. 2015; Hilty et al. 2018), social media (privacy, security, professionalism) (Hilty et al. 2018; Zalpuri et al. 2018), mobile health (CDS, mobile phones, apps) (Hilty et al. 2019a, b, 2020), and asynchronous care (procedures, resource utilization, role in system navigation and workflow, and accountability, ethics, and responsibility) competencies (Torous and Roberts 2017; Hilty et al. 2020). The CTiBS (Hilty et al. 2017; Maheu et al. 2018), clinical informatics (Torous et al. 2017), and the Clinical Informatics Milestones (ACGME 2013) were also helpful.
Within patient care, the subdomains are patient assessment/evaluation, engagement and interpersonal skills, management and treatment planning, administration and medico-legal issues, procedures, and clinical decision-making, synthesis, and information systems (IS). One challenge is optimizing the match between the sensor and a specific application context, so a structured approach for requirements and evaluation would help a novice/advanced beginner. A healthy middle-aged adult engaging in employee wellness in a large organization will have different needs than a 70-year-old at home suffering from depression (Caulfield et al. 2019). They require different devices, indifferent sensor locations (wrist versus wearable vest), steps for synchronization (from own device to data aggregator), and a software change (from “reliable” to validated measurement). This may proceed smoothly if the healthcare system prepares options, has staff to assist, and trains faculty supervisors.
Also in patient care, a novice/advanced beginner should understand what a device is used for (e.g., behavior, physiology), its capacity (i.e., connection with databases), weigh the value of data, monitor/adjust treatment with supervision, and help patients reflect on use and output. A supervisor would need to adjust teaching, supervision/availability, and workflow (e.g., between visit documentation) based on the technology, learner skill level, and patient care outcomes. The competent/proficient clinician would base the selection of a wearable on patient preference, skill, and need (i.e., purpose), and show proficiency when moving between regular to synchronous, continuous care. They need to remember that the therapeutic frame includes in-person, video, and mobile health engagement, in order to create a positive virtual environment.
Under professionalism, subdomains include attitude, accountability, integrity and ethics, reflection, feedback, and growth. A novice/advanced beginner would be open to learning and adhere to professional guidelines (i.e., timely response to text, documentation). Learning boundaries may come naturally and require small but important adjustments (e.g., learning to see if self-assessments match data reports). A competent/proficient clinician would understand boundaries/ethical dilemmas with technological options and how to engage patients (e.g., sensitively inquire how it feels to have self-assessments double-checked with in-time data or behavior). If symptoms like suicidal thoughts or worsening depression appear linked to a particular geographic setting, the clinician may ask if the patient has noticed a pattern before simply announcing a conclusion. An advanced/expert clinician would provide leadership, teaching, and consultation, including assessing/advising if clinical responsibilities shift as care models changes to ensure practice within license and regulations (e.g., how often clinicians are expected to check data inflow and make decisions).
Teaching, Evaluation and Assessment: Clinical Care and Supervision in Training
Teaching sensor and wearable competencies successfully will require a mix of methods layered and adjusted for increasing skill level over time. This requires evaluation methods that can determine whether competencies have been achieved and for supervisors to provide formative feedback to learners (Table 4) (Tekian et al. 2015). The following case examples illustrate approaches to the teaching and supervising sensor and wearable competencies.
Within the patient care, one subdomain is patient assessment/evaluation. An academic medical center may decide to offer a weight management program to support patient goals by collecting sensor data on step count via an app. Novice/advanced beginners must be able to screen for technology use among eligible patients, educate patients on the use of technology, and monitor/adjust treatment with supervision. The competent/proficient clinician would teach in the curricular for trainees, seminars for interprofessional teams, and on supervisory rounds. A case or simulation could introduce concepts to learners and be linked to clinical workflow. The advanced/expert clinician would develop the educational approach, train teachers, and evaluate process of change, aggregate sensor data, and on clinical quality and cost-effectiveness outcomes.
Within the patient care, the clinical decision-making, synthesis, and IS subdomain focus on EHR and documentation (Table 3). A novice/beginner should effectively read and write in the EHR, consider CDS tools, and use tools to coordinate and document key care events (Fig. 2). If a sensor is inputting data into the EHR between appointments, will the trainee know to review it or understand prompts? Can they adjust treatment plans “in time”, or do they need the supervisor to adjust practices (Table 4)? Typically, trainees wait and review data in a 1-h, scheduled, weekly supervision of a caseload, but “curbside consultations” are frequently needed. This could work better with two, half-hour meetings, more “walk-in” supervision time or more flexible coverage policies (see professional reflection, monitoring and hygiene related to patient care and supervision and subdomain in-time supervision in-person or at distance on critical incident). A competent/proficient clinician should select a wearable to ensure good usability, durability, interactivity (e.g., chat, feedback), and fit with the EHR for decision-making and documentation. They educate patients, trainees, and staff on tools’ pros/cons, and the meaning of the data, and like supervisors, has to consider questions like, “What is reasonable checking of data between visits? What standard of care is sought for?”
Case-based learning (Table 4) uses real life examples or vignettes in seminar, site-based case conferences and other presentations. Trainee’s experience with patients could be presented in a weekly group supervision or site-based case conference with an interprofessional team. Interactive methods like role-play using peers can highlight issues, practice communication skills, identify options for decisions, and propose solutions for patients. This provides an opportunity to build and/or solidify the trainee role as an educator (Table 4), working with an interprofessional team and adapting communication skills to multiple people.
The original telepsychiatric competencies suggested that institutions develop an approach for video and other technologies on the e-health (or e-behavioral health) spectrum (Hilty et al. 2015). The IT-business model (Hilty et al. 2019; Hilty et al. 2019; Ray et al. 2007) provides six steps for institutions to implement new technologies: (1) assess readiness; (2) create/hardwire the culture; (3) write policies and procedures; (4) establish the curriculum and competencies; (5) train learners and faculty; and (6) evaluate/manage change. One literature review focused on additional suggestions for synchronous and asynchronous telehealth (Hilty et al. 2020) and organized institutional competencies into 5 focus areas: patient-centered care; evaluation and outcomes; roles/needs of participants (e.g., trainees, faculty, teams, professions); teams, professions and systems within institutions; and the academic health center institutional structure, process, and administration.
A variety of sensors and wearables are being used today and they are creating new options for patient care, clinician decision-making, and population health. These options reduce geographical, cost, and temporal barriers (Naslund et al. 2016; Tachakra et al. 2003; Cowan et al 2019) and provide an opportunity to bring patients and clinical teams together for communication, support, and intervention (Kumari et al. 2017; Torous and Roberts 2017). Successful integration of these technologies into care requires a person- and patient-centered approach, with patients moving from mere passengers to valued, responsible drivers of their care. Clinicians also must be aware of required skills, knowledge, and training to assure ethical and competent practice.
Our review of established guidelines and position statements highlights several important questions and concerns. Professional organizations have prioritized and made suggestions on how to integrate video, phones and e-mail, but mobile health (e.g., apps, texts) and wearable sensors are underrepresented or not mentioned at all. Nor are boards, agencies, and other government sectors showing significant movement on mobile health technologies that correspond with current and emerging paradigm shifts.
Research on sensors and wearables, remote monitoring, coupled with video, social media, mobile health, and asynchronous competencies, provides a foundation for the competencies we identified in this review. The domains and subdomains are a starting place for further discussion, elaboration, and improvement. The training approach with cases and curricular options we presented provides a source of options for clinicians, teachers, and program directors to help others develop knowledge, skills, and attitudes and has worked for other complex endeavors (e.g., culture and diversity, rural health) (Gifford et al. 2012; Office of Rural Health and Department of Veterans Affairs Employee Education System 2017; Hilty et al. 2018; Hilty et al. 2018; Kurtz 2018). Methodologies are needed to develop, implement, and evaluate (quantitative, qualitative) clinically and competency-based interventions (Proctor et al. 2010; Curran et al. 2012; Gargon et al. 2019; Marcolino et al. 2019; Solino-Fernandez et al. 2019). These learner-centered approaches parallel patient-centered ones and ensure quality care via technology (Hilty et al. 2020). Institutional competencies for synchronous and asynchronous technologies (Hilty et al. 2019; Hilty et al. 2020) could guide the paradigm shift for sensors and wearables for behavioral health in primary, integrated, and community care settings (Hilty et al. 2018; Hilty et al. 2019; Winters et al. 2019; Hilty et al. 2020).
Wearable, sensors, and other remote monitoring technologies are providing opportunities for additional data collection and sharing that have the potential to improve care. Moreover, traditional applications of clinical decision support technology that have existed for decades, such as electronic health records and expert systems, are being enhanced by newer technologies, such as cloud computing, artificial intelligence (AI), ML, and algorithms to provide access to new data that help clinicians constantly reevaluate, intervene, and make new observations and decisions (Luxton 2016; Patel 2002). Clinicians and those who train clinicians must embrace these technological advancements as part of healthcare reform so students’ experiences with technology enhance digital professionalism (Mostaghimi et al. 2017) and health systems can foster an e-culture with a wide range of evidence-based technology options (Hilty et al. 2019; Hilty et al. 2020). Computing tools can improve structure, process, and synergy with the way clinicians’ approach their work, particularly as research and clinical trials build upon empirical foundations and standardize methods, interventions, and evaluation (Armstrong et al. 2017; Bush et al. 2019).
Some organizations may be reluctant to adapt to change in general and specifically related to technology (Hilty et al. 2015). This includes academic health centers that have to balance missions with partner institutions, regional health campuses, and many health networks. Health systems are still working on basic clinical, technological, and administrative workflows, so most have not prioritized competencies. Their buy-in is needed but requires integration of missions and institutional change. Development and use of competencies for any clinical care change is an ongoing process (Holmboe, Sherbin, Donlin, Swing and Frank 2010), and the first steps involve discussion, needs assessment, implementation, and evaluation from faculty and program directors.
Even if competencies are prioritized, there are other competing priorities within psychiatric, medical, and interprofessional education (Hilty et al. 2015; Sockalingham et al. 2018), each requiring faculty development for teaching, supervision, and evaluation (Steinert et al. 2016). Some clinicians, training directors, and healthcare leaders familiar with telehealth are concerned about patient safety, information quality, professionalism, informed consent, privacy, and other medico-legal matters related to sensors and wearables. What they may fail to realize is that these and other risks are already in play with basic technologies (e.g., e-mail, phone call), growing proportionately with others (e.g., apps, texts, digital image transfer) and exponentially with sensors and wearables. Thus, the easier, more direct way to approach these issues may be to build an IT-business-health model, with input and collaboration by diverse partners such as clinicians, teachers, informaticians, and technology (i.e., developers, engineers).
Health care and medicine incrementally change over time based on social, political and scientific roots (Sturmberg et al. 2014). New paradigms of teaching and clinical care are needed to shape institutional response to innovative shifts technology disruptions and organizational contradictions that complicate institutional change (Bowe et al. 2003). A consolidated effort by training directors, national organizations, accreditation agencies, medical boards, and government could make change happen (Hilty et al. 2015). The result may be a change in the model of care, funding, and reimbursement beyond the in-person/video model and underserved populations, consistent with the call to make the temporary waivers during the COVID-19 pandemic permanent.
Across healthcare settings, training leaders, researchers, clinical champions, and technology specialists should collaborate to focus priorities on a general set of technology competencies that could be integrated into each profession’s existing set. Such a framework may avoid traditional pitfalls of trying to add completely new competencies, confusion about where they fit, concern about too many levels, and more generic pedagogic distinctions (e.g., competencies versus activities/tasks knowledge versus skills/behaviors) (Fairchild and Bayer 2004; Hilty et al. 2015).
Limitations of this paper include the scoping review style that was used that examines the extent, range, and nature of research in a topic area and identifies gaps in knowledge, rather than examine more specific, narrow topics as with systematic reviews (Arksey and O’Malley 2005; Levac et al. 2010). Second, each of the concept areas is unique and complex, so greater depth of analysis is needed to determine the “what, when, why and how” they work—and the human factors for patients and teams with varying skill levels and backgrounds. Third, country/province, state/federal, and healthcare institutions are unique, so one approach may not fit all systems. Fourth, core competencies must be clearly defined, implemented, and evaluated for each clinical specialty, in conjunction with professional development, change management, and building of a positive e-culture for participants to succeed.
Research is promising for sensors and wearables, and it is shifting toward standardized methods, interventions, and evaluation measures. As these technologies are integrated into health care, steps are needed to ensure quality care, clinicians’ skills, and outcomes that are in line with in-person and synchronous telehealth care. Clinician skillsets for sensors and wearables are similar to and overlap with those required for in-office or telehealth-based care, but there are additional components. Clinician and institutional competencies need to be implemented and evaluated to ensure quality of care, integration of missions, and institutional change. Attention to clinical, educational, administrative and technical factors is needed to build a positive e-culture of care, and collaboration is needed between clinicians, teachers, informaticians, developers, and engineers.
Abdullah, S., Matthews, M., Frank, E., Doherty, G., Gay, G., & Choudhury, T. (2016). Automatic detection of social rhythms in bipolar disorder. Journal of the American Medical Informatics Association, 23, 538–543.
Accreditation Council on Graduate Medical Education. (2013). Common program requirements. Retrieved from: https://www.acgme.org/acgmeweb/Portals/0/PFAssets/ProgramRequirements/CPRs2013.pdf.
American Association for Marriage and Family Therapy. (2015). Standard VI technology-assisted professional services. Retrieved from: https://www.aamft.org/iMIS15/AAMFT/Content/legal_ethics/code_of_ethics.aspx.
American Association for Marriage and Family Therapy Regulatory Board. (2016). Teletherapy guidelines. Retrieved from: https://amftrb.org/wp-content/uploads/2017/05/Proposed-Teletherapy-Guidelines-DRAFT-as-of-09.12.16.pdf.
American Counseling Association. (2014). Code of ethics. Retrieved from: https://www.counseling.org/resources/aca-code-of-ethics.pdf.
American Psychiatric Association and American Telemedicine Association. (2018). Best practices in videoconferencing-based telemental health. Retrieved from: https://www.psychiatry.org/psychiatrists/practice/telepsychiatry/blog/apa-and-ata-release-new-telemental-health-guide.
American Psychiatric Association. (2017). Resource document on telemedicine: synchronous videoconferencing in psychiatry. Retrieved from: https://www.resource-document-telemedicine-synchronous-video-conferencing-in-psychiatry-2017.
American Psychiatric Association. (2014). Resource document on telepsychiatry and related technologies in clinical psychiatry. Retrieved from: https://www.Resource-2014-Telepsychiatry-Clinical-Psychiatry.
American Psychological Association Educational Directorate. (2012). A practical guidebook for the competency benchmarks. Retrieved from: http://www.apa.org/ed/graduate/benchmarks-guide.aspx.
American Psychological Association. (2013). Guideline for the practice of telepsychology. Retrieved from: http://www.apapracticecentral.org/ce/guidelines/telepsychology-guidelines.pdf.
American Psychological Association. (2019). Proposed guidelines for the optimal use of social media in professional psychological practice. In Press.
American Telemedicine Association. (2013). ATA practice guidelines for video-based online mental health services. Telemedicine Journal and E-Health, 19(9), 722–330.
American Telemedicine Association. (2017). Practice guideline for child and adolescent telemental health. Telemedicine Journal and E-Health. Retrieved from: https://higherlogicdownload.s3.amazonaws.com/AMERICANTELEMED/618da447-dee1-4ee1-b941-c5bf3db5669a/UploadedImages/Practice%20Guideline%20Covers/NEW_ATA%20Children%20&%20Adolescents%20Guidelines.pdf.
American Telemedicine Association. (2014). Practice guidelines for live, on demand primary and urgent care. Retrieved from: http://www.medicalinfo.ch/images/articles/ATA-Practice-Guidelines-for-Live-On-Demand-Primary-and-Urgent-Care.pdf.
American Telemedicine Association. (2009). Practice guidelines for videoconferencing-based telemental health. Retrieved from: http://www.americantelemed.org/docs/default-source/standards/practice-guidelines-for-videoconferencing-based-telemental-health.pdf?sfvrsn=6.
Areàn, P. A., Ly, K. H., & Andersson, G. (2016). Mobile technology for mental health assessment. Dialogues in Clinical Neuroscience, 18(2), 163–169.
Ariga, K., Makita, T., Ito, M., Mori, T., Watanabe, S., & Takeya, J. (2019). Review of advanced sensor devices employing nanoarchitectonics concepts. Beilstein Journal of Nanotechnology, 10, 2014–2030.
Arksey, H., & O’Malley, L. (2005). Scoping studies: towards a methodological framework. International Journal of Social Research Methodology, 8(1), 19–32.
Armontrout, J., Torous, J., Fisher, M., Drogin, E., & Gutheil, T. (2016). Mobile mental health: navigating new rules and regulations for digital tools. Current Psychiatry Reports, 18(10), 91. https://doi.org/10.1007/s11920-016-0726-x.
Armstrong, C. M., Hoyt, T., Kinn, J. T., Ciulla, R. P., & Bush, N. E. (2017). Mobile behavioral health applications for the military community: evaluating the emerging evidence base. Best Practices in Mental Health: Mental Health Practice with Military and Veteran Connected Populations, 13(1), 106–119, Vandiver & Corcoran (Editors). ISSN 1553–555x.
Association of State and Provincial Psychology Boards. (2014). Maintenance of competence for licensure (MOCL) white paper. Retrieved from: https://cdn.ymaws.com/asppb.site-ym.com/resource/resmgr/Guidelines/Maintenance_of_Competence_fo.pdf.
Australian Psychological Society. (2011). Guidelines for providing psychological services and products using the internet and telecommunications technology. Retrieved from: https://aaswsocialmedia.wikispaces.com/file/view/EG-Internet.pdf.
Baldominos, A., & Quintana, D. (2019). Data-driven interaction review of an ed-tech application. Sensors, 19, 1910.
Balon, R., Beresin, E. V., Coverdale, J. H., Louie, A. K., & Roberts, L. W. (2015). Strengthening telepsychiatry’s role in clinical care and education. Academic Psychiatry, 39(1), 6–9.
Barisone, M., Bagnasco, A., Timmins, F., Aleo, G., & Sasso, L. (2018). Approaches to nurse education and competence development in remote telemonitoring of heart failure patients with implanted heart devices in Italy: a cause for concern. European Journal of Cardiovascular Nursing, 17(5), 388–389.
Bennett, C. C., Doub, T. W., & Selove, R. (2012). EHRs connect research and practice: where predictive modeling, artificial intelligence, and clinical decision support inter- sect. Health Policy and Technology, 1(2), 105–114.
Bowe, C. M., Lahey, L., Kegan, R., & Armstrong, E. (2003). Questioning the “big assumptions.” Part II: recognizing organizational contradictions that impede institutional change. Medical Education, 37(8), 723–733.
British Association for Counselling and Psychotherapy. (2019). Working online in the counselling professions. Retrieved from: https://www.bacp.co.uk/media/2162/bacp-working-online-supplementary-guidance-gpia047.pdf.
Bush, N. E., Armstrong, C. M., & Hoyt, T. V. (2019). Smartphone apps for psychological health: a brief state of the science review. Psychological Services, 16(2), 188–195.
Canadian Counselling and Psychotherapy Association. (2019). Technology and innovative solutions, chapter project. Guidelines for uses of technology in counselling and psychotherapy. Retrieved from: https://www.ccpa-accp.ca/wp-content/uploads/2019/04/TISCGuidelinesMar2019EN.pdf.
Canadian Psychological Association. (2006). Providing psychological services via electronic media (draft ethical guidelines). Retrieved from: http://www.cpa.ca/aboutcpa/committees/ethics/psychserviceselectronically/.
Cardinale, M., & Varley, M. C. (2017). Wearable training-monitoring technology: applications, challenges, and opportunities. International Journal of Sports Physiology and Performance, 12, S255–S262. https://doi.org/10.1123/ijspp.2016-0423.
Caulfield, B., Reginatto, B., & Slevin, P. (2019). Not all sensors are created equal: a framework for evaluating human performance measurement technologies. NPJ Digital Medicine, 2, 7. https://doi.org/10.1038/s41746-019-0082-4.
Clark, A. (2013). Whatever next? Predictive brains, situated agents, and the future of cognitive science. Behavioral and Brain Sciences, 36(3), 181–204.
Cowan, K., Gentry, M., McKean, A., & Hilty, D. M. (2019). Barriers to use of telepsychiatry: clinicians as gatekeepers. Mayo Clinic Proceedings, 94(12), 2510–2523.
Crawford, A., Sunderji, N., Lopez, J., & Soklaridis, S. (2016). Defining competencies for the practice of telepsychiatry through an assessment of resident learning needs. BMC Medical Education, 16, 28. https://doi.org/10.1186/s12909-016-0529-0.
Curran, G. M., Bauer, M., Mittman, B., Pyne, J. M., & Stetler, C. (2012). Effectiveness-implementation hybrid designs: combining elements of clinical effectiveness and implementation research to enhance public health impact. Medical Care, 50(3), 217–226. https://doi.org/10.1097/MLR.0b013e3182408812.
Dinh-Le, C., Chuang, R., Chokshi, S., & Mann, D. (2019). Wearable health technology and electronic health record integration: scoping review and future directions. Journal of Medical Internet Research mhealth uhealth, 7(9), e12861. https://doi.org/10.2196/12861.
Dogan, E., Sander, C., Wagner, X., Hegerl, U., & Kohls, E. (2017). Smartphone-based monitoring of objective and subjective data in affective disorders: where are we and where are we going? Systematic review. Journal of Medical Internet Research, 19(7), e262. https://doi.org/10.2196/jmir.7006.
Dreyfus, S. E. & Dreyfus, H. L. (1980). A five-stage model of the mental activities involved in directed skill acquisition. Berkeley, CA: University of California, Operations Research Center. Retrieved from: https://www.dtic.mil/cgi-bin/GetTRDoc?AD=ADA084551.
Elgendi, M., & Menon, C. (2019). Assessing anxiety disorders using wearable devices: challenges and future directions. Brain Sci, 9(3), 50. https://doi.org/10.3390/brainsci9030050.
Fairchild, A. L., & Bayer, R. (2004). Public health: ethics and the conduct of public health surveillance. Science, 303(5658), 631–632. https://doi.org/10.1126/science.1094038.
Faurholt-Jepsen, M., Munkholm, K., Frost, M., Bardram, J. E., & Kessing, L. V. (2016). Electronic self-monitoring of mood using IT platforms in adult patients with bipolar disorder: a systematic review of the validity and evidence. BMC Psychiatry, 16(1), 7. https://doi.org/10.1186/s12888-016-0713-0.
Ferguson, J. E., & Redish, A. D. (2011). Wireless communication with implanted medical devices using the conductive properties of the body. Expert Review of Medical Devices, 8(4), 427–433. https://doi.org/10.1586/erd.11.16.
Gaglani, S. M., & Topol, E. J. (2014). iMedEd: the role of mobile health technologies in medical education. Academic Medicine, 89(9), 1207–1209. https://doi.org/10.1097/ACM.0000000000000361.
Garcia-Ceja, E., Riegler, M., Nordgreen, T., Jakobsen, P., Oedegaard, K. J., & Tørresen, J. (2018). Mental health monitoring with multimodal sensing and machine learning: a survey. Pervasive Mobile Computing, 51, 1–26.
Gargon, E., Gorst, S. L., & Williamson, P. R. (2019). Choosing important health outcomes for comparative effectiveness research: 5th annual update to a systematic review of core outcome sets for research. PLoS ONE, 14(12), e0225980. https://doi.org/10.1371/journal.pone.0225980.
Gifford, V., Niles, B., Rivkin, I., Koverola, C., & Polaha, J. (2012). Continuing education training focused on the development of behavioral telehealth competencies in behavioral healthcare providers. Rural and Remote Health, 12(2108), 1–15.
Glover, J. A., Williams, E., Hazlett, L. J., & Campbell, N. (2013). Connecting to the future: telepsychiatry in postgraduate medical education. Telemedicine Journal and E-Health, 19(6), 474–479.
Godleski, L. (2012). A comprehensive national telemental health training program. Academic Psychiatry, 36(5), 408–410.
González-Crespo, R., & Burgos, D. (2019). Advanced sensors technology in education. Sensors (Basel), 19(19), 4155. https://doi.org/10.3390/s19194155.
Goss, S., Anthony, K., & Jamieson, A. (2001). Guidelines for online counselling and psychotherapy. Lutterworth, UK: British Association for Counselling and Psychotherapy.
Greenes, R. A., Bates, D. W., Kawamoto, K., Middleton, B., Osheroff, J., & Shahar, Y. (2018). Clinical decision support models and frameworks: seeking to address research issues underlying implementation successes and failures. Journal of Biomedical Informatics, 78, 134–143.
Grekin, E. R., Beatty, J. R., & Ondersma, S. J. (2019). Mobile health interventions: exploring the use of common relationship factors. Journal of Medical Internet Research mhealth uhealth, 7(4), e11245. https://doi.org/10.2196/11245.
Harrison, R., Flood, D., & Duce, D. (2013). Usability of mobile applications: literature review and rationale for a new usability model. Journal of Interactive Science, 1(1), 1–16.
Heikenfeld, J., Jajack, A., Rogers, J., Gutruf, P., Tian, L., Pan, T., ... & Kim, J. (2018). Wearable sensors: modalities, challenges, and prospects. Laboratory Chip, 18, 217–248.
Hermens, H. J., & Vollenbroek-Hutten, M. M. (2008). Towards remote monitoring and remotely supervised training. Journal of Electromyography and Kinesiolology, 18(6), 908–919. https://doi.org/10.1016/j.jelekin.2008.10.004.
Hersh, W. R., Gorman, P. N., Biagioli, F. E., Mohan, V., Gold, J. A., & Mejicano, G. C. (2014). Beyond information retrieval and electronic health record use: competencies in clinical informatics for medical education. Advances in Medical Education and Practice, 1(5), 205–212.
Hetrick, S. E., Robinson, J., Burge, E., Blandon, R., Mobilio, B., Rice, S. M., ... & Davey, C. (2018). Youth codesign of a mobile phone app to facilitate self-monitoring and management of mood symptoms in young people with major depression, suicidal ideation, and self-Harm. Journal of Medical Internet Research Mental Health, 5(1), e9. https://doi.org/10.2196/mental.9041.
Hilty, D. M., Armstrong, C. M., Luxton, D. D., Gentry, M., & Krupinski, E. (2020). A scoping review of sensors, wearables and remote monitoring for behavioral health: uses, outcomes, clinical competencies and research directions. Journal of Technology in Behavioral Science: In Press.
Hilty, D. M., Chan, S., Torous, J., Luo, J., & Boland, R. J. (2019a). A telehealth framework for mobile health, smartphones and apps: competencies, training and faculty development. Journal of Technology in Behavioral Science, 4(2), 106–123.
Hilty, D. M., Chan, S., Torous, J., Luo, J., & Boland, R. J. (2019b). A competency-based framework for psych/behavioral health apps for trainees, faculty, programs and health systems. Psychiatric Clinics in North America, 42, 513–534.
Hilty, D. M., Chan, S., Torous, J., Luo, J., & Boland, R. J. (2020). A framework for competencies for the use of mobile technologies in psychiatry and medicine. Journal of Medical Internet Research uhealth and mhealth, 8(2) http://mhealth.jmir.org/2020/2/e12229/.
Hilty, D. M., Chan, S., Torous, J., Mahautmr, J., & Mucic, D. M. (2015). New frontiers in healthcare and technology: Internet- and web-based mental options emerge to complement in-person and telepsychiatric care options. Journal of Health and Medical Informatics, 6(4), 1–14.
Hilty, D. M., Crawford, A., Teshima, J., Chan, S., Sunderji, N., & Yellowlees, P. M. (2015). A framework for telepsychiatric training and e-health: competency-based education, evaluation and implications. International Review of Psychiatry, 27(6), 569–592.
Hilty, D. M., Evangelatos, G., Valasquez, A., & Sosa, J. (2018). Telebehavioral health for rural culturally diverse populations: approaches for clinical services, competencies and administration. Journal of Technology in Behavioral Science, 3, 206–220.
Hilty, D. M., Feliberti, J., Evangelatos, G., Lu, F. G., & Lim, R. F. (2018). Competent cultural telebehavioral healthcare to diverse populations: administration, evaluation, and financing. Journal of Technology in Behavioral Science, 4(3), 186–200.
Hilty, D. M., Ferrer, D., Johnston, B., Johnston, B., Callahan, E. J., & Yellowlees, P. M. (2013). The effectiveness of telemental health: a 2013 review. Telemedicine Journal and E-Health, 19(6), 444–454.
Hilty, D. M., Gentry, M, McKean, A. J., Cohen, K. E., Lim, R. F., Lu, F. G. (2020). Telehealth for rural diverse populations: telebehavioral and cultural competencies, clinical outcomes and administrative approaches. mHealth, 6, 20. https://doi.org/10.21037/mhealth.2019.10.04.
Hilty, D. M., Maheu, M., Drude, K. P., Hertlein, K. M., Wall, K., Long, R. P., & Luoma, T. L. (2017). Telebehavioral health, telemental health, e-therapy and e-health competencies: the need for an interprofessional framework. Journal of Technology in Behavioral Science, 2, 171–189. https://doi.org/10.1007/s41347-017-0036-0.
Hilty, D. M., Maheu, M., Drude, K., & Hertlein, K. M. (2018). The need to implement and evaluate telehealth competency frameworks to ensure quality care across behavioral health professions. Academic Psychiatry, 42(6), 818–824. https://doi.org/10.1007/s40596-018-0992-5.
Hilty, D. M., Nesbitt, T. S., Marks, S. L., & Callahan, E. J. (2002). How telepsychiatry affects the doctor-patient relationship: communication, satisfaction, and additional clinically relevant issues. Primary Psychiatry, 9(9), 29–34.
Hilty, D. M., Parish, M., Chan, S., Torous, J., Xiong, G., Scher, L., & Yellowlees, P. M. (2020). A comparison of in-person, synchronous and asynchronous telepsychiatry: skills/competencies, teamwork and administrative workflow. Journal of Technology in Behavioral Science. https://doi.org/10.1007/s41347-020-00137-8.
Hilty, D. M., Rabinowitz, T. R., McCarron, R. M., Katzelnick, D. J., Chang, T., Bauer, A. M., & Fortney, J. (2018). Telepsychiatry and e-mental health models leverage stepped, collaborative, and integrated services to primary care. Psychosomatics, 59(3), 227–250.
Hilty, D. M., Randhawa, K., Maheu, M. M., McKean, A., Pantera, R., Mishkind, M. C., & Rizzo, A. (2020). A review of telepresence, virtual reality and augmented reality applied to clinical care. Journal of Technology in Behavioral Science, 5, 178–205.
Hilty, D. M., Sunderji, N., Suo, S., Chan, S., & McCarron, R. M. (2019). Telepsychiatry and other technologies for integrated care: evidence-base, best practice models and competencies. International Review of Psychiatry, 1, 1–18.
Hilty, D. M., Torous, J., Parish, M., Chan, S., Xiong, G., Scher, L., & Yellowlees, P. M. (2020). A literature review comparing clinicians’ approaches and skills to in-person, synchronous and asynchronous care: moving toward asynchronous competencies to ensure quality care. Telemedicine Journal and E-Health. https://doi.org/10.1089/tmj.2020.0054.
Hilty, D. M., Uno, J., Chan, S., Torous, J., & Boland, R. J. (2019). Role of technology in professional development. Psychiatric Clinics in North America, 42, 493–512.
Hilty, D. M., Unützer, J., Ko, D. G., Luo, J., Worley, L., & Yager, J. (2019). Approaches for departments, schools, and health systems to better implement technologies used for clinical care and education. Academic Psychiatry, 43(6), 611–616. https://doi.org/10.1007/s40596-019-01074-2.
Hilty, D. M., Zalpuri, I., Stubbe, D., Snowdy, C. E., Shoemaker, E. Z., Myint, M. T., ... & Liu, H. Y. (2018). Social media/networking as part of e-behavioral health and psychiatric education: competencies, teaching methods, and implications. Journal of Technology in Behavioral Science, 3(4), 268–293.
Hoffman, P., & Kane, J. M. (2015). Telepsychiatry education and curriculum development in residency training. Academic Psychiatry, 39(1), 108–109.
Holmboe, E. S., Sherbin, J., Donlin, M. L., Swing, S. R., & Frank, J. R. (2010). The role of assessment in competency-based medical education. Medical Teacher, 32, 676–682. https://doi.org/10.3109/0142159X.2010.500704.
Iobst, W. F., Sherbino, J., Cate, T. O., Richardson, D. L., Dath, D., Swing, S. R., ... & Frank, J. R. (2010). Competency-based medical education in postgraduate medical education. Medical Teacher, 32(8), 651–656.
Joshi, S. V., Stubbe, D., Li, S. T., & Hilty, D. M. (2019). Correction to the use of technology by youth: implications for psychiatric educators. Academic Psychiatry, 43(4), 466. https://doi.org/10.1007/s40596-018-1013-4.
Kemp, J., Zhang, T., Inglis, F., Wiljer, D., Sockalingam, S., Crawford, A., ... & Strudwick, G. (2020). Delivery of compassionate mental health care in a digital technology–driven age: scoping review. Journal of Medical Internet Research, 22(3), e16263. https://doi.org/10.2196/16263.
Khan, S., & Ramtekkar, U. (2019). Child and adolescent telepsychiatry education and training. Psychiatric Clinics in North America, 42(4), 555–562.
Kumari, P., Mathew, L., & Syal, P. (2017). Increasing trend of wearables and multimodal interface for human activity monitoring: a review. Biosensors and Bioelectronics, 90, 298–307.
Kurtz, D., Janke, R., Vinek, J., Wells, T., Wells, T., Hutchinson, P., & Froste, A. (2018). Health sciences cultural safety education in Australia, Canada, New Zealand, and the United States: a literature review. International Journal of Medical Education, 9, 271–285.
Kutafina, E., Laukamp, D., & Jonas, S. M. (2015). Wearable sensors in medical education: supporting hand hygiene training with a forearm EMG. Studies in Health Technology and Informatics, 211, 286–291.
Kvedar, J., Coye, M. J., & Everett, W. (2014). Connected health: a review of technologies and strategies to improve patient care with telemedicine and telehealth. Health Affairs (Millwood), 33(2), 194–199.
Levac, D., Colquhoun, H., & O’Brien, K. K. (2010). Scoping studies: advancing the methodology. Implemenation Science, 5, 69. https://doi.org/10.1186/1748-5908-5-69.
L’Hommedieu, M., L’Hommedieu, J., Begay, C., Schenone, A., Dimitropoulou, L., Margolin, G., et al. (2019). Lessons learned: recommendations for implementing a longitudinal study using wearable and environmental sensors in a health care organization. Journal of Medical Internet Research mhealth uhealth, 7(12), e13305. https://doi.org/10.2196/13305.
Li, H., Wu, J., Gao, Y., & Shi, Y. (2016). Examining individuals’ adoption of healthcare wearable devices: an empirical study from privacy calculus perspective. International Journal of Medical Informatics, 88, 8–17. http://www.ncbi.nlm.nih.gov/pubmed/26878757.
Lobelo, F., Kelli, H. M., Tejedor, S. C., Pratt, M., McConnell, M. V., Martin, S. S., & Welk, G. J. (2016). The wild wild west: a framework to integrate mhealth software applications and wearables to support physical activity assessment, counseling and interventions for cardiovascular disease risk reduction. Progress in Cardiovascular Disease, 58(6), 584–594. https://doi.org/10.1016/j.pcad.2016.02.007.
Loncar-Turukalo, T., Zdravevski, E., Silva, J., Chouvarda, I., & Traijkovik, V. (2019). Literature on wearable technology for connected health: scoping review of research trends, advances, and barriers. Journal of Medical Internet Research, 21(9), e14017. https://doi.org/10.2196/14017.
Luxton, D. D., Kayl, R. A., & Mishkind, M. C. (2012). mHealth data security: the need for HIPAA-compliant standardization. Telemedicine Journal and E-Health, 18, 284–288.
Luxton, D. D. (2016). Artificial intelligence in behavioral health care. Boston, MA: Elsevier.
Ma, Y., Xu, B., Bai, Y., Sun, G., & Zhu, R. (2014). Infer daily mood using mobile phone sensing. Ad Hoc Sensor Wireless Network, 20, 133–152.
Mackintosh, K. A., Chappel, S. E., Salmon, J., Timperio, A., Ball, K., Brown, H., ... & Ridgers, K. D. (2019). Parental perspectives of a wearable activity tracker for children younger than 13 years: acceptability and usability study. Journal of Medical Internet Research mhealth uhealth, 7(11), e13858. https://doi.org/10.2196/13858.
Maddison, R., Cartledge, S., Rogerson, M., Goedhart, N. S., Ragbir Singh, T., Neil, C., ... & Ball, K. (2019). Usefulness of wearable cameras as a tool to enhance chronic disease self-management: scoping review. Journal of Medical Internet Research mhealth uhealth, 7(1), e10371. https://doi.org/10.2196/10371.
Maheu M., Drude, K., Hertlein, K., Lipschutz, R., Wall, K., Long, R., et al., & Hilty, D. M. (2019). An interdisciplinary framework for telebehavioral health competencies. Journal of Technology in Behavioral Science, 3(2), 108–40; correction 3(2):107. https://doi.org/10.1007/s41347-019-00113.
Maheu, M., Drude, K., Hertlein, K. M., & Hilty, D. M. (2018). A framework for interprofessional telebehavioral health competencies: implementation and challenges moving forward. Academic Psychiatry, 42(6), 825–833. https://doi.org/10.1007/s40596-018-0988-1.
Marcolino, M. S., Alkmim, M. B., Pessoa, C. G., Maia, J. X., & Cardoso, C. S. (2019). Development and implementation of a methodology for quality assessment of asynchronous teleconsultations. Telemedicine Journal and E-Health, 26(5). https://doi.org/10.1089/tmj.2019.0049.
Mermelstein, H., Krupinski, E., DeGuzman, E., Rabinowitz, T., & Hilty, D. M. (2017). The application of technology to health: the evolution of telephone to telemedicine and telepsychiatry: a historical review and look at human factors. Journal of Technology in Behavioral Science, 1(2), 5–20.
Metter, D. F., Woolley, F. R., Bradley, Y., Morrison, J., Phillips, W. T., Salman, U., ... & Dodd, G. D, 3rd (2006). Teaching radiology resident didactics using videoconferencing development of a videoconferencing network to increase the efficiency of radiology resident training. Academic Radiology, 13(10), 1276–1285. https://doi.org/10.1016/j.acra.2006.06.023.
Mohr, D., Zhang, M., & Schueller, S. (2017). Personal sensing: understanding mental health using ubiquitous sensors and machine learning. Annual Review of Clinical Psychology, 13, 23–47. https://doi.org/10.1146/annurev-clinpsy-032816-044949.
Mostaghimi, A., Olszewski, A. E., Bell, S. K., Roberts, D. H., & Crotty, B. H. (2017). Erosion of digital professionalism during medical students’ core clinical clerkships. Journal of Medical Internet Research Medical Education, 3(1), e9. https://doi.org/10.2196/mededu.6879.
Naslund, J. A., Aschbrenner, K. A., & Bartels, S. J. (2016). Wearable devices and smartphones for activity tracking among people with serious mental illness. Mental Health and Physical Activity, 10, 10–17. https://doi.org/10.1016/j.mhpa.2016.02.001.
National Academy of Sciences, Engineering, and Medicine (2020). Developing a patient-centered approach to optimizing veterans access to health care services. Retrieved from: https://www.nationalacademies.org/event/03-12-2020/developing-a-patient-centered-approach-to-optimizing-veterans-access-to-health-care-services-a-workshop.
National Association of Social Workers. (2017). Association of social work boards, council on social work education, and clinical social work association, standards for technology in social work practice. Retrieved from: https://www.naswpress.org/publications/standards/technology.html.
Office of Rural Health and Department of Veterans Affairs (2017). Employee education system, Veterans Health Administration Office of Care Coordination services. Cultural competence in telehealth clinics. Retrieved from: https://www.ruralhealth.va.gov/docs/cultural-competence-in-telehealth-clinics.pdf.
Patel, V. L., Kaufman, D. R., & Arocha, J. F. (2002). Emerging paradigms of cognition in medical decision-making. Journal of Biomedical Informatics, 35(1), 52–75.
Piwek, L., Ellis, D. A., Andrews, S., & Joinson, A. (2016). The rise of consumer health wearables: promises and barriers. PLoS Medicine, 13, e1001953. https://doi.org/10.1371/journal.pmed.1001953.
Proctor, E., Silmere, H., Raghavan, R., Raghavan, R., Hovmand, P., Aarons, G., ... & Hensley, M. (2010). Outcomes for implementation research: conceptual distinctions, measurement challenges, and research agenda. Administrative Policy in Mental Health, 38(2), 65–76.
Ramkumar, P. N., Haeberle, H. S., Bloomfield, M. R., Schaffer, J. L., Kamath, A. T., Patterson, B. M., & Krebs, V. E. (2019). Artificial intelligence and arthroplasty at a single institution: real-world applications of machine learning to big data, value-based care, mobile health, and remote patient monitoring. Journal of Arthroplasty, 34(10), 2204–2209. https://doi.org/10.1016/j.arth.2019.06.018.
Ravizza, A., De Maria, C., Di Pietro, L., Sternini, F., Audenino, A. L., & Bignardi, C. (2019). Comprehensive review on current and future regulatory requirements on wearable sensors in preclinical and clinical testing. Frontiers in Bioengineering and Biotechnology, 7, 313. https://doi.org/10.3389/fbioe.2019.00313.
Ray, G., Muhanna, W. A., & Barney, J. B. (2007). Competing with IT: the role of shared IT-business understanding. Communications of the ACM, 50(12), 87–91.
Rohani, D. A., Faurholt-Jepsen, M., Kessing, L. V., & Bardram, J. E. (2018). Correlations between objective behavioral features collected from mobile and wearable devices and depressive mood symptoms in patients with affective disorders: systematic review. Journal of Medical Internet Research mhealth uhealth, 6(8), e165. https://doi.org/10.2196/mhealth.9691.
Roque, F., Cechinel, C., Weber, T. O., Lemos, R., Villarroel, R., Miranda, D., & Munoz, R. (2019). Using depth cameras to detect patterns in oral presentations: a case study comparing two generations of computer engineering students. Sensors, 19, 3493. https://doi.org/10.3390/s19163493.
Royal College of Physicians and Surgeons Canada. (2015). CanMEDS physician competency framework. Retrieved from: http://canmeds.royalcollege.ca/uploads/en/framework/CanMEDS%202015%20Framework_EN_Reduced.pdf
Russell, M. A., & Gajos, J. M. (2020). Annual research review: ecological momentary assessment studies in child psychology and psychiatry. Journal of Child Psychology and Psychiatry, 61(3), 376–394. https://doi.org/10.1111/jcpp.13204.
Sano, A., Phillips, A. J., Yu, A. Z., McHill, A. W., Taylor, S., Jaques, N., et al. (2015). Recognizing academic performance, sleep quality, stress level, and mental health using personality traits, wearable sensors and mobile phones. International Conference on Wearable and Implantable Body Sensor Networks. https://doi.org/10.1109/BSN.2015.7299420.
Seko, Y., Kidd, S., Wiljer, D., & McKenzie, K. (2014). Youth mental health interventions via mobile phones: a scoping review. Cyberpsycholology Behavioral and Social Networking, 17(9), 591–602.
Seppälä, J., De Vita, I., Jämsä, T., Miettunen, J., Isohanni, M., Rubinsten, K., ... & Bulgheroni, M. (2019). Mobile phone and wearable sensor-based mhealth approaches for psychiatric disorders and symptoms: systematic review. Journal of Medical Internet Research Mental Health, 6(2), e9819. https://doi.org/10.2196/mental.9819.
Silva, B., Rodrigues, J., Diez, I., Lopez-Coronado, M., & Saleem, K. (2015). Mobile-health: a review of current state in 2015. Journal of Biomed Informatics, 56, 265–272.
Sockalingam, S., Arena, A., Serhal, E., Mohri, L., Alloo, J., & Crawford, A. (2018). Building provincial mental health capacity in primary care: an evaluation of a project ECHO mental health program. Academic Psychiatry, 42(4), 451–457.
Solino-Fernandez, D., Ding, A., Bayro-Kaiser, E., & Ding, E. L. (2019). Willingness to adopt wearable devices with behavioral and economic incentives by health insurance wellness programs: results of a US cross-sectional survey with multiple consumer vignettes. BMC Public Health, 19(1), 1649. https://doi.org/10.1186/s12889-019-7920-9.
Statista. (2018). Connected wearable devices worldwide 2016–2021. Retrieved from: https://www.statista.com/statistics/487291/global-connected-wearable-devices/.
Steinert, Y., Mann, K., Anderson, B., Barnett, B. M., Centeno, A., Naismith, L., ... & Dolmans, D. (2016). A systematic review of faculty development initiatives designed to enhance teaching effectiveness: a 10-year update: BEME Guide No. 40. Medical Teacher, 38(8), 769–786. https://doi.org/10.1080/0142159X.2016.1181851.
Steinhubl, S. R., Muse, E. D., & Topol, E. J. (2013). Can mobile health technologies transform health care? JAMA, 310(22), 2395–2396.
Sturmberg, J. P., O’Halloran, D., Colagiuri, R., Fernandez, A., Lukersmith, S., Torkfar, G., & Salvador-Carulla, L. (2014). Health care frames - from Virchow to Obama and beyond: the changing frames in health care and their implications for patient care. Journal of Evaluation and Clinical Practice, 20(6), 1036–1044.
Sunderji, N., Crawford, A., & Jovanovic, M. (2015). Telepsychiatry in graduate medical education: a narrative review. Academic Psychiatry, 39(1), 55–62.
Tachakra, S., Wang, X. H., Istepanian, R. S., & Song, Y. H. (2003). Mobile e-health: the unwired evolution of telemedicine. Telemedicine Journal and E-Health, 9(3), 247–257.
Tekian, A., Hodges, B. D., Roberts, T. E., Schuwirth, L., & Norcini, J. (2015). Assessing competencies using milestones along the way. Medical Teacher, 37(4), 399–402.
Teshima, J., Hodgins, M., Boydell, K. M., & Pignatiello, A. (2016). Resident evaluation of a required telepsychiatry clinical experience. Academic Psychiatry, 40(2), 348–352.
Thompson, W. G., Kuhle, C. L., Koepp, G. A., McCrady-Spitzer, S. K., & Levine, J. A. (2014). Go4Life exercise counseling, accelerometer feedback, and activity levels in older people. Archives of Gerontology and Geriatrics, 58(3), 314–319.
Torous, J., & Roberts, L. W. (2017). The ethical use of mobile health technology in clinical psychiatry. Journal of Nervous and Mental Disorders, 205(1), 4–8. https://doi.org/10.1097/nmd.0000000000000596.
Torous, J., Chan, S., Luo, J., Boland, R., & Hilty, D. (2017). Clinical informatics in psychiatric training: preparing today’s trainees for the already present future. Academic Psychiatry, 42(5), 694–697.
Wahle, F., Kowatsch, T., Fleisch, E., Rufer, M., & Weidt, S. (2016). Mobile sensing and support for people with depression: a pilot trial in the wild. Journal of Medical Internet Research mhealth uhealth, 4(3), e111. https://doi.org/10.2196/mhealth.5960.
Wang, Q. E., Myers, M. D., & Sundaram, D. (2013). Digital natives and digital immigrants towards a model of digital fluency. Business and Information Systems Engineering, 5(6), 409–419.
Winters, N., Langer, L., Nduku, P., Robson, J., O’Donovan, J., Maulik, P., ... & Nagraj, S. (2019). Using mobile technologies to support the training of community health workers in low-income and middle-income countries: mapping the evidence. BMJ Global Health, 4(4), e001421. https://doi.org/10.1136/bmjgh-2019-001421.
Winters, N., Langer, L., & Geniets, A. (2018). Scoping review assessing the evidence used to support the adoption of mobile health (mHealth) technologies for the education and training of community health workers (CHWs) in low-income and middle-income countries. British Medical Journal Open, 8(7), e019827. https://doi.org/10.1136/bmjopen-2017-019827.
Witt, D. R., Kellogg, R. A., Snyder, M. P., & Dunn, J. (2019). Windows into human health through wearables data analytics. Current Opinion in Biomedical Engineering, 9, 28–46.
Yellowlees, P., Parish, M. B., González, Á., Chan, S., Hilty, D., Iosif, A., ... & Xiong, G. (2018). Asynchronous telepsychiatry: a component of stepped integrated care. Telemedicine Journal and E-Health, 24(5), 375–378.
Zalpuri, I., Liu, H., Stubbe, D., Wrozek, M., Hadsu, J., & Hilty, D. M. (2018). A competency-based framework for social media for trainees, faculty and others. Academic Psychiatry, 42(6), 808–817.
Accreditation Council of Graduate Medical Education, American Telemedicine Association and Telemental Health Interest Group, Northern California Veterans Administration Health Care System, UC Davis School of Medicine, Office of Connected Care, Department of Veterans Affairs, Defense Health Agency, Department of Defense.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Hilty, D.M., Armstrong, C.M., Edwards-Stewart, A. et al. Sensor, Wearable, and Remote Patient Monitoring Competencies for Clinical Care and Training: Scoping Review. J. technol. behav. sci. 6, 252–277 (2021). https://doi.org/10.1007/s41347-020-00190-3
- Mobile health