Developing a Digitally Informed Curriculum in Psychiatry Education and Clinical Practice

Technology is rapidly permeating the lives of our patients and influencing the clinical practice of psychiatry [1]. From electronic health record (EHR) systems to patient-focused smartphone apps, the way healthcare professionals communicate information with other professionals and with patients have been reshaped dramatically in the last decade [2]. This is partly due to increased access to smartphones [3], and partly due to nationwide initiatives to modernize the healthcare system with a goal to increase accuracy and efficiency. With increased access to technology, patients and their families have become proactive in obtaining information, rather than relying solely on education from their healthcare providers. Social media use also serves as an alternate avenue in obtaining medical information, including subscribing to interest groups and connecting to others with similar medical conditions [4, 5].

Millennial students and trainees in medicine have also developed proficiency and, even more so, preference for incorporating technology in their practice, learning, and daily lives [6]. However, they cite lack of guidance in education as a major barrier in incorporating technology fully in their practice [7]. Elements of teaching technology for child and adolescent psychiatry [8], rural and underserved psychiatry [9], and general adult psychiatry have been described. Despite demonstrated efficacy and desire among the early career physicians to incorporate technology in their practice, there is a notable discrepancy in culture and expectation of restraint around use of technology in various settings between generations [10].

To further formalize the practice of incorporating technology in health systems and clinical practice, the Accreditation Council for Graduate Medical Education (ACGME) has accredited a new fellowship, named Clinical Informatics. To date, there are currently 26 Clinical Informatics fellowships across the country [11]. This new subspecialty enables candidates from any medical specialty to become experts in the informatics and analytics of medical care [12]. Despite the clear trend of increased technology use and need for defined competencies on this topic, there remains a lack of education in technology in general psychiatry and other psychiatry subspecialty trainings despite the rising number of issues associated with rapid infiltration of technology in our daily lives and thus our daily practice.

In this article, we aim to explore the current state of technology education and the impact of technology on mental health, and discuss a pilot curriculum in providing a framework for evaluating various technologies in practice. To achieve such aims, the objectives of this article include (a) to develop a conceptual approach related to technology for clinical care, (b) to grasp the factors that drive the use of technology services and expected outcomes of its many users related to health, communication, and care, and (c) to discuss case examples of technology use utilizing the conceptualized approach.

The Changing Landscape of Medical Education, Healthcare Delivery, and Information Access

Before creating a curriculum, we must understand the current technology landscape and challenges among clinicians and patients. A reasonable interpretation of the relative lag in technology curriculum can be attributed to the ambivalent feelings towards the rapid advancement and infiltration of technology in multiple domains of life, including the field of psychiatry.

The benefits of technology are experienced and utilized by clinicians daily. Well-known web-based sources such as UpToDateR and LexicompR provide busy practitioners virtually instant access to well-validated medical information. This access is paralleled among patients through websites such as WebMD, Mayo Clinic, and . With these resources, physicians have increased the ability to educate patients both inside and outside the office. Technology has allowed psychiatrists, patients, and their families to access information with a single click or tap. Such convenience among patients and consumers has raised concerns regarding the validity of information, which could be harmful to uninformed users.

Technology has also permeated into clinicians’ professional and career development, such as online Continuing Medical Education (CME). Through the internet, providers can attend online lectures remotely and earn credits. Psychiatrist have also used technology to share ideas and collaborate virtually through software and connected devices, including teleconferencing, cloud drives, and shared online notebooks. EHR systems also have significantly changed the way important health information is shared or recorded by clinicians. Within EHR systems, communication with patients has also diversified into the email, communication portals, and apps. EHR systems often incorporate electronic prescription systems [13]; once incorporated, these systems have helped reduce medication prescribing errors versus traditional paper-based prescribing [14].

In addition to increased access to resources, continuing medical education, and changes in communication with patients, technology has also increased patients’ access to psychiatry through telemedicine. Through introduction and utilization of telepsychiatry, our specialty has broadened our care delivery radius and increased the availability of psychiatric evaluation, especially in rural or sub-urban areas that lack psychiatric care [15,16,17]. Similarly, non-psychiatric providers now also have increased access options to psychiatric consultation for their psychiatric patients, particularly with integrated care and collaborative care settings [18,19,20]. Despite relatively lacking an evidence base, many mobile device apps are publicly available and target psychiatric disorders such as anxiety [21], social anxiety [22], depression, posttraumatic stress disorder, and obsessive compulsive disorder [23]. Technology-assisted devices such as FitbitR and Verily’s Study Watch are being tested in clinical trials and continue to push the envelope of incorporating technology in psychiatric care.

Computers can not only provide this increased access but also provide novel methods of assessing one’s symptoms. Wearable devices—such as computer-equipped smart watches, fitness tracking devices, and jewelry with embedded sensors—can track the wearer’s physical activity and physiology. For instance, wrist-worn actigraphy can measure psychomotor retardation, an indication of apathy in depressive disorders; psychomotor agitation, an indication of movement disorders; and sleep, a key component in mood disorders [24]. Ecological momentary assessment methods allow patients to report symptoms and answer questionnaires from within interactive apps without having to wait to answer such questions in a clinical setting, allowing in-the-moment assessments and reducing issues of recall and underreporting [25,26,27,28,29].

Assessing symptoms can provide not only novel forms of patient data but also novel ways to influence clinical decisions. Providers can use clinical decision support systems (CDSS) to help them make sense of clinical and research data, reduce medical errors, and increase efficiencies. CDSS has been implemented in EHR systems to inform best practices—such as the use of antidepressant medications in the Texas Medication Algorithm Project (TMAP) [30, 31]—and avoid patient safety errors, such as avoiding medication overdoses and interactions [32]. Research studies can take advantage of CDSS by alerting the provider—at defined times for defined, eligible patients—that a patient is eligible for a clinical trial, in the use of Clinical Trial Alerts (CTA) [33, 34].

Finally, technology can provide new ways of providing interventions, such as through virtual reality (VR) and augmented reality (AR). These technologies immerse the patients in a new environment using displays that partially or completely cover their field of view. VR has particularly been studied most for exposure therapy to treat anxiety and trauma disorders, but research to date has been of low quality and quantity [35,36,37,38,39].

Although there are several notable positives in technology in healthcare, there are notable negatives as well. There are several potential concerns when using technology in psychiatry. One example that stands out is the lack of regulatory policies in place that protect consumers when using apps for their mental health. Most of these apps lack or do not show any evidence to support their claim [21,22,23, 40] and the potential harm can be hidden from the average users. Among the most notable concerns within app use is the vulnerability to security and privacy violations.

EHRs are equally vulnerable. In 2017, NHS was hacked and vast amount of personal health data was compromised [41, 42]. In some commercial companies, not only are there unverified, misleading, or even potentially dangerous information made available through technology, but health user data are frequently mined and sold to corporations without actively notifying their users [43]. Social media can also affect sleep, anxiety, and depression among Scottish adolescents. What they found was that adolescents with high usage or night time usage showed poorer sleep quality and higher incidents of anxiety and depression [44]. These major concerns of technology are known, yet patients and their families often are not informed or educated about the topic. This can be attributed to the lack of awareness among patients and lack of guidance and education from their providers.

Having a digitally informed curriculum can help providers use and integrate technology into behavioral health treatments, manage provider-patient relationships with technology, and be better equipped to treat the health of populations. [45,46,47]

Understanding the current status quo in technology education is imperative in designing the pilot curriculum. The Psychiatry Milestone Project under ACGME lists a few competency domains that could be directly or indirectly applicable to issues related to technology in psychiatry (The Psychiatry Milestone Project 2013). When searching for the term “technology” within the milestones, only “ICS2. Information sharing and record keeping” specifically comments on the expected proficiency and milestones for psychiatry trainees. Within that milestone, it sets the expectation that trainees must be able to ensure that the written record that includes electronic medical records (EMR) are accurate and timely. However, there is no specific guideline on how education on EMRs should be conducted. Thus, trainees are trained specifically to suit the needs of their teaching institution and state licensing board. Further, there are no specific milestones set for ethics, professional and patient communication, and the general practice of psychiatry in relation to technology and devices outside of pharmacotherapy, electroconvulsive therapy (ECT), and transcranial magnetic stimulation (TMS). Thus, it becomes infinitely difficult for programs to develop a curriculum that could improve trainee understanding of technology, which further perpetuates the hesitance of utilizing technology among psychiatry trainees.

Recognizing Current Clinical Challenges

It can be challenging to develop a comprehensive curriculum due to the rapidly changing landscape of technology and digital psychiatry. An important point to consider is that technology is dynamic and advances with rapid speed that can cause unforeseen issues and opportunities. Recognizing the challenge, the American Psychiatric Association created a task force on app evaluation. Within the taskforce, Torous et al. proposed a dynamic framework as an evaluating tool and guideline for practitioners, rather than utilizing a static scoring system that makes keeping up with the speed of technological advances challenging [48]. Using the app evaluation tool as a guide, the authors have conceptualized a digitally informed curriculum that covers three main topics: hardware, software, and the digital environment.

The first topic, hardware, is a critical first step for psychiatry, akin to understanding the importance of ECT, TMS, radiological imaging devices, etc. GPS and digital tracking devices have become ubiquitous, and are being researched and deployed as psychiatric assessment tools and interventions. Smartphones and devices can automatically track patients using embedded sensors that are aware of medication ingestion, the environment, and their location; patients can also manually enter in their own data to track and monitor psychiatric symptoms [49, 50]. Studies have demonstrated benefits of using sensory tracking system in diagnosing and monitoring psychiatric symptoms [51]. Although this study relied heavily in the coding and interpretation of the programming, it would not have been possible without the reliability of GPS tracking within smartphone devices. Recently, the US Federal Drug Administration (FDA) approved a modified form of aripiprazole medication which embeds a sensor [52]. This sensor reports if the medication was ingested, thus helping patients and providers monitor compliance with medications. Although a clinician may not need to know the exact engineering of the device, they will need basic education on understanding a device’s capabilities, similar to how clinicians must understand basic biochemistry and pharmacology before prescribing a medication.

Second, understanding the software can help providers more effectively use and recommend software resources. Mobile device apps are a dominant form of software in which patients interact with technology daily. Given the rapidity of app creation and app expiration in the marketplace, it makes education on specific apps unsustainable. Further, the evidence in using apps on specific mental health conditions remains limited when compared to psychopharmacology or interventional procedures. Recently, reSETR, developed by Pear therapeutics, has been approved for substance use disorders by the FDA, that is used to complement a larger treatment ecosystem [53]. Until there is further research in app efficacy and mainstream use of apps as a clinical tool, it would be extremely challenging to teach specific apps for clinical care in the same way we educate on pharmacology such as the mechanism of action and efficacy. Artificial Intelligence (AI) and its most widely used form, machine learning, are another major topic not only in psychiatry but also in medicine in general due to its huge implications for promises and concerns. A growing number of studies rely on or utilize machine learning [54, 55]. Contrary to many, studies have reported that patients could be more comfortable or encouraged to report their symptoms to so-called “Conversational Artificial Intelligence” [56]. In a recent study, utilization of online computerized cognitive behavioral therapy (CCBT) was associated with better outcome than that of usual care in primary care settings [57]. Meanwhile, many potential dangers have been associated with use of AI, such as replacement of human jobs and misuse and control of AI by people with mal intention [58], and have been alarming to clinicians and patients. However, further studies are required before more precautions or promises are warranted.

Third, providers must understand systems, including the broad digital environment and non-clinical factors that influence clinical care and patient health. The use of clinical and practice tools—EHR, e-prescribing, CDSS, and digital presence management—along with patient-accessible consumer tools—social media, VR, AR, smart devices, and smartphones—must be understood in the context of legal and ethical issues. EHR systems are no longer new to any clinicians who have trained in the twenty-first century, but clinical competency and understanding of EHR systems vary. For some clinicians, the widespread use of EHRs has also generated concerns such as increased documentation burden, decreased face to face time with patients, due to this documentation demand, and risks of privacy breaches. Telepsychiatry through video encounters is also becoming more mainstream. However, many clinicians are forced to learn about this system in an ad hoc fashion, such as county, state, and federal rules and regulations [59,60,61]. Social media is another digital ecosystem that has infiltrated our lives and is becoming an alternate avenue for medical professionals to reach out to the masses. As of 2015, Pew Research Center reported that 65% of all adults, including 89% of those aged 18 to 34, use at least one social networking site [62, 63]. These platforms have allowed more people to connect and network, but also have presented many concerns. More studies show that the use of social media is significantly linked to depression and reportedly contributing to the increased incidence of depression in children, adolescent, and young adults in the twenty-first century [64, 65]. Social media also have presented unprecedented challenges, such as live broadcasting suicide attempts and online bullying. When these problems present into the office, clinicians may feel underprepared to address them because of a lack of a digitally informed curriculum.

Developing a Digitally Informed Curriculum for Psychiatric Training

With these three main topics as a focus, the authors present the reformulated framework into a Digitally Informed Pyramid (DIP) for clinicians of any level of training to approach technology (see Fig. 1). The principle is the same: break down the construct of the technology and approach it from a clinical adaptive lens. In this curriculum, we propose the DIP as a model to be used in a short didactic course in training, as well as a daily tool in psychiatry practice. This framework is designed to allow educators and students to learn the foundation in a systematic way, but also enables them to apply it dynamically across their daily practice.

Fig. 1

The Digitally Informed Pyramid (DIP)—adapted from the APA app pyramid

Our proposed framework takes form of a pyramid that encourages users to work from the bottom to the top, with the principle that not all technology will progress to the top of the pyramid as they must pass basic expectations and have a certain foundation. These levels are Background, Risk, Evidence, Feasibility, and Communication. First, like in any clinical scenario, the clinician must assess the background and understand the context of the technology presented. Then, the clinician should assess the potential risk when using a specific technology. Next, the clinician should seek the evidence supporting the clinical use of a particular technology. Once the evidence supporting its use is explored, a clinician will examine the feasibility of implementing this technology towards the clinical practice whether it is for education or for treatment. Lastly, communication to patients and families is emphasized after the evaluation process.

In our proposed curriculum, we recommend combining didactics with case vignettes to help facilitate and maximize learning among trainees and practicing psychiatrist. Within this curriculum, we recommend covering the evidence, risks, and feasibility of the aforementioned technology domains while pairing them with vignettes described below. When discussing the case vignettes, consider challenging participants to describe their own experiences, fears and roadblocks at each level. For example, when discussing the Background level, encourage participants to openly share and discuss daily encountered problems and challenges regarding technology in patients’ or their own lives. By doing so, participants are encouraged to learn to identify and understand the context within which each unique problem or challenge is presented.

In the second level of the pyramid, Risks, trainees will use their knowledge to assess potential risks and issues associated within a given scenario or technology. These could be a combination of ethical issues, legal risks, and cultural considerations. The Evidence level will challenge trainees to seek evidence that is available and relevant to their current clinical scenario. It is important to keep in mind that current clinical evidence has not supported presumed benefits or expected risks among available technology and encourage trainees and clinicians to be critical regarding their potential biases when forming a clinical hypothesis. Formulating the Feasibility level encourages trainees to consider realistic and clinical application of the considered technology. With patients as the participants’ focus, this step is essential in making sure their proposed solutions or recommendations accommodate patients’ lives and abilities. The fifth and last level, Communication, is a discussion on how trainees can best communicate their recommendations to their patients, families, and their colleagues. When implementing the proposed curriculum, it is important to remember that sessions can be combined or expanded depending on the needs of the participants. Similar to the pyramid, it is a suggested framework that can be dynamic to suit the needs of the educator and learner (Tables 1 and 2).

Table 1 Example curriculum based on the Digitally Informed Pyramid (DIP)
Table 2 Key points within each level of Digitally Informed Pyramid (DIP)

Applying the Digitally Informed Pyramid to Clinical Vignettes

Didactic complemented with case-based learning has proved to be the best approach for adult learners. The following are a few clinical vignettes aimed to demonstrate the intended use of Digitally Informed Pyramid (DIP) for informed discussion with patients.

  • Vignette 1: A 40-year-old male patient with history of schizophrenia and frequent medication non-compliance presents to clinic expressing interest in trying medication with tracking device to help with compliance. He voices hesitance around how the tracking device may affect his GI health and would like further education on the topic (Tables 3).

  • Vignette 2: A psychiatry intern uses her smartphone to look up a journal article on PubMed during a didactic course. The lecturing attending admonishes her for using her phone during a class (Tables 4).

  • Vignette 3: A 15-year-old female patient presents with depression. The patient reveals several months history of cyberbullying by unknown peers, which has exacerbated her symptoms significantly (Tables 5).

Table 3 Example of DIP for Vignette 1
Table 4 Example of DIP for Vignette 2
Table 5 Example of DIP for Vignette 3

Future Directions

The Digitally Informed Pyramid (DIP) was created with intent to provide a dynamic framework to not only allow educators to introduce students and trainees to technology related issues in psychiatry, but also empower clinicians to facilitate an informed discussion with their patients. The authors recognize that DIP is not exhaustive and may not address all available technology. Thus, ongoing research and pilot trials of implementing a technology curriculum are warranted to further understand the best approach to prepare psychiatrists in managing technology in practice. Also, more research is needed with focus on how to teach technology, who should teach it, when and where it is taught, and how different topics are organized and grouped—for instance, EHR and decision support vs. apps and wearables vs. VR and AR.

Also, challenges are anticipated for most training programs, if not all, in developing and implementing new didactic courses within their tightly maintained schedules. In addition, due to innate variabilities within respective residency training programs, it would be difficult or nearly impossible to standardize and assure a good quality of such didactic curriculum across the country. Therefore, the authors suggest that ACGME develop and propose a new set of competencies pertaining to technology in psychiatry as well as other medical specialties accordingly. Although the focus of this article is on psychiatry training, the need for technology-oriented curriculum under structured guidelines by ACGME is believed to be ubiquitous in medicine. Timely ACGME guidelines on knowledge and skills for trainees to achieve during their residency trainings would also encourage and enable more programs to develop and implement adequate technology curriculum that is warranted and feasible in their respective field.


  1. 1.

    Luxton DD, June JD, Kinn JT. Technology-based suicide prevention: current applications and future directions. Telemed J E Health. 2011;17(1):50–4.

    Article  Google Scholar 

  2. 2.

    Tavares J, Oliveira T. Electronic health record patient portal adoption by health care consumers: an acceptance model and survey. J Med Internet Res. 2016;18(3):e49.

    Article  Google Scholar 

  3. 3.

    Poushter J. Smartphone ownership and internet usage continues to climb in emerging economies. 2016 9/22/2016 [cited 2017 9/15/2017]; Available from:

  4. 4.

    Kantor D, Bright JR, Burtchell J. Perspectives from the patient and the healthcare professional in multiple sclerosis: social media and patient education. J Neurol Ther. 2017.

    Article  Google Scholar 

  5. 5.

    Benetoli A, Chen TF, Aslani P. Consumer health-related activities on social media: exploratory study. J Med Internet Res. 2017;19(10):e352.

    Article  Google Scholar 

  6. 6.

    Hilty DM. The present and future interface of technology and medical education. Acad Psychiatry. 2006;30(6):437–8.

    Article  Google Scholar 

  7. 7.

    Gipson S, Torous J, Boland R, Conrad E. Mobile phone use in psychiatry residents in the United States: multisite cross-sectional survey study. JMIR Mhealth Uhealth. 2017;5(11):e160.

    Article  Google Scholar 

  8. 8.

    Gipson SY-MT, Kim JW, Shin AL, Kitts R, Maneta E. Teaching child and adolescent psychiatry in the twenty-first century. Child and Adolescent Psychiatric Clinics. 2017;26(1):93–103.

    Article  Google Scholar 

  9. 9.

    Chung-Do J, Helm S, Fukuda M, Alicata D, Nishimura S, Else I. Rural mental health: implications for telepsychiatry in clinical service, workforce development, and organizational capacity. Telemed e-Health. 2012;18(3):244–46.

    Article  Google Scholar 

  10. 10.

    Hilty DM, Belitsky R, Cohen MB, Cabaniss DL, Dickstein LJ, Bernstein CA, et al. Impact of the information age on residency training: communication, access to public information, and clinical care. Acad Psychiatry. 2015;39(1):104–7.

    Article  Google Scholar 

  11. 11.

    AMIA. Clinical Informatics Fellowship Programs. [cited 2017 21 Dec]; Available from:

  12. 12.

    Torous J, Chan S, Luo J., et al. Clinical informatics in psychiatric training: preparing today’s trainees for the already present future. Acad Psychiatry. 2017.

    Article  Google Scholar 

  13. 13.

    Bell DS, Friedman MA. E-prescribing and the medicare modernization act of 2003. Health Aff (Millwood). 2005;24(5):1159–69.

    Article  Google Scholar 

  14. 14.

    Abramson EL, Pfoh ER, Barrón Y, Quaresimo J, Kaushal R. The effects of electronic prescribing by community-based providers on ambulatory medication safety. Jt Comm J Qual Patient Saf. 2013;39(12):545–52.

    Article  Google Scholar 

  15. 15.

    Hilty DM, Cobb HC, Neufeld JD, Bourgeois JA, Yellowlees PM. Telepsychiatry reduces geographic physician disparity in rural settings, but is it financially feasible because of reimbursement? Psychiatr Clin N Am. 2008;31(1):85–94.

    Article  Google Scholar 

  16. 16.

    Hilty DM, Yellowlees PM, Cobb HC, Neufeld JD, Bourgeois JA. Use of secure e-mail and telephone: psychiatric consultations to accelerate rural health service delivery. Telemed e-Health. 2006;12(4):490–5.

    Article  Google Scholar 

  17. 17.

    Hilty DM, Yellowlees PM, Sonik P, Derlet MN, Hendren RL. Rural child and adolescent telepsychiatry: successes and struggles. Pediatr Ann. 2009;38(4).

  18. 18.

    Hilty DM, Servis ME, Nesbitt TS, Hales RE. The use of telemedicine to provide consultation-liaison service to the primary care setting. Psychiatr Ann. 1999;29(7):421–7.

    Article  Google Scholar 

  19. 19.

    Hilty DM, Yellowlees PM, Cobb HC, Bourgeois JA, Neufeld JD, Nesbitt TS. Models of telepsychiatric consultation—liaison service to rural primary care. Psychosomatics. 2006;47(2):152–7.

    Article  Google Scholar 

  20. 20.

    Hilty DM, Yellowlees PM. Collaborative mental health services using multiple technologies: the new way to practice and a new standard of practice? J Am Acad Child Adolesc Psychiatry. 2015;54(4):245–6.

    Article  Google Scholar 

  21. 21.

    Sucala M, Cuijpers P, Muench F, Cardoș R, Soflau R, Dobrean A, et al. Anxiety: there is an app for that. A systematic review of anxiety apps. Depress Anxiety. 2017;34(6):518–25.

    Article  Google Scholar 

  22. 22.

    Alyami M, Giri B, Alyami H, Sundram F. Social anxiety apps: a systematic review and assessment of app descriptors across mobile store platforms. Evid Based Ment Health. 2017;20(3):65–70.

    Article  Google Scholar 

  23. 23.

    Van Ameringen M, et al. There is an app for that! The current state of mobile applications (apps) for DSM-5 obsessive-compulsive disorder, posttraumatic stress disorder, anxiety and mood disorders. Depress Anxiety. 2017;34(6):526–39.

    Article  Google Scholar 

  24. 24.

    Vahia IV, Sewell DD. Late-life depression: a role for accelerometer technology in diagnosis and management. Am J Psychiatry. 2016;173(8):763–8.

    Article  Google Scholar 

  25. 25.

    Schueller SM, Aguilera A, Mohr DC. Ecological momentary interventions for depression and anxiety. Depression Anxiety. 2017;34(6):540–5.

    Article  Google Scholar 

  26. 26.

    Myin-Germeys I, Klippel A, Steinhart H, Reininghaus U. Ecological momentary interventions in psychiatry. Curr Opin Psychiatry. 2016;29(4):258–63.

    Article  Google Scholar 

  27. 27.

    Arean PA, Hoa Ly K, Andersson G. Mobile technology for mental health assessment. Dialogues Clin Neurosci. 2016;18(2):163–9.

    PubMed  PubMed Central  Google Scholar 

  28. 28.

    Marszalek J, et al. Using ecological momentary assessment to evaluate current physical activity. Biomed Res Int. 2014;2014:915172.

    Article  Google Scholar 

  29. 29.

    Moskowitz DS, Young SN. Ecological momentary assessment: what it is and why it is a method of the future in clinical psychopharmacology. J Psychiatry Neurosci: JPN. 2006;31(1):13–20.

    PubMed  Google Scholar 

  30. 30.

    Kurian BT, Trivedi MH, Grannemann BD, Claassen CA, Daly EJ, Sunderajan P. A computerized decision support system for depression in primary care. Prim Care Companion J Clin Psychiatry. 2009;11(4):140–6.

    Article  Google Scholar 

  31. 31.

    Trivedi MH, Kern JK, Grannemann BD, Altshuler KZ, Sunderajan P. A computerized clinical decision support system as a means of implementing depression guidelines. Psychiatr Serv. 2004;55(8):879–85.

    Article  Google Scholar 

  32. 32.

    Berner ES, La Lande TJ. Overview of clinical decision support systems, in clinical decision support systems: theory and practice. In: Berner ES, editor. Cham: Springer International Publishing; 2016. p. 1–17.

    Google Scholar 

  33. 33.

    Adler-Milstein J, Embi PJ, Middleton B, Sarkar IN, Smith J. Crossing the health IT chasm: considerations and policy recommendations to overcome current challenges and enable value-based care. J Am Med Inform Assoc. 2017;24(5):1036–43.

    Article  Google Scholar 

  34. 34.

    Embi PJ, Leonard AC. Evaluating alert fatigue over time to EHR-based clinical trial alerts: findings from a randomized controlled study. J Am Med Inform Assoc. 2012;19(e1):e145–8.

    Article  Google Scholar 

  35. 35.

    Maples-Keller JL, Bunnell BE, Kim SJ, Rothbaum BO. The use of virtual reality Technology in the Treatment of anxiety and other psychiatric disorders. Harv Rev Psychiatry. 2017;25(3):103–13.

    Article  Google Scholar 

  36. 36.

    McCann RA, Armstrong CM, Skopp NA, Edwards-Stewart A, Smolenski DJ, June JD, et al. Virtual reality exposure therapy for the treatment of anxiety disorders: an evaluation of research quality. J Anxiety Disord. 2014;28(6):625–31.

    Article  Google Scholar 

  37. 37.

    Motraghi TE, Seim RW, Meyer EC, Morissette SB. Virtual reality exposure therapy for the treatment of posttraumatic stress disorder: a methodological review using CONSORT guidelines. J Clin Psychol. 2014;70(3):197–208.

    Article  Google Scholar 

  38. 38.

    Goncalves R, et al. Efficacy of virtual reality exposure therapy in the treatment of PTSD: a systematic review. PLoS One. 2012;7(12):e48469.

    CAS  Article  Google Scholar 

  39. 39.

    Opris D, et al. Virtual reality exposure therapy in anxiety disorders: a quantitative meta-analysis. Depress Anxiety. 2012;29(2):85–93.

    Article  Google Scholar 

  40. 40.

    Kim JW, Nguyen TQ, Gipson S Y-M T, Shin AL, Torous J. Smartphone apps for autism spectrum disorder—understanding the evidence. J Technol Behav Sci. 2017.

    CAS  Article  Google Scholar 

  41. 41.

    The Telegraph. Government under pressure after NHS crippled in global cyber attack as weekend of chaos looms. 2017 [cited 2017 17 Dec]; Available from:

  42. 42.

    The Telegraph. NHS cyber attack: Everything you need to know about ‘biggest ransomware’ offensive in history. 2017 [cited 2017 17 Dec]; Available from:

  43. 43.

    Consumer Reports. Consumers Less Confident About Healthcare, Data Privacy, and Car Safety, New Survey Finds. 2017 [cited 2017 17 Dec]; Available from:

  44. 44.

    Woods HC, Scott H. #Sleepyteens: Social media use in adolescence is associated with poor sleep quality, anxiety, depression and low self-esteem. J Adolesc. 2016;51:41–9.

    Article  Google Scholar 

  45. 45.

    Chan S, Godwin H, Gonzalez A, Yellowlees PM, Hilty DM. Review of use and integration of mobile apps into psychiatric treatments. Curr Psychiatry Rep. 2017;19(12):96.

    Article  Google Scholar 

  46. 46.

    Parish MB, Fazio S, Chan S, Yellowlees PM. Managing psychiatrist-patient relationships in the digital age: a summary review of the impact of technology-enabled care on clinical processes and rapport. Curr Psychiatry Rep. 2017;19(11):90.

    Article  Google Scholar 

  47. 47.

    Chan S, Torous J, Hinton L, Yellowlees P. Mobile tele-mental health: increasing applications and a move to hybrid models of care. Healthcare. 2014;2(2):220–33.

    Article  Google Scholar 

  48. 48.

    Zagorski N. Psychiatric News. APA to Provide Framework to Evaluate Mobile Health Apps. 2017 [cited 2017 6 Jan]; American Psychiatric Association Available from:

  49. 49.

    Chan S, Torous JB, Hinton L, Yellowlees PM. Psychiatric apps: patient self-assessment, communication, and potential treatment interventions. In: Mucic D, Hilty D, editors. e-Mental Health. Springer Cham; 2016.

    Google Scholar 

  50. 50.

    Torous JB, Chan SR, Yellowlees PM, Boland R. To use or not? Evaluating ASPECTS of smartphone apps and mobile technology for clinical care in psychiatry. J Clin Psychiatry. 2016;77(6):e734–8.

    Article  Google Scholar 

  51. 51.

    Or F, Torous J, Onnela JP. High potential but limited evidence: using voice data from smartphones to monitor and diagnose mood disorders. Psychiatr Rehabil J. 2017;40(3):320–4.

    Article  Google Scholar 

  52. 52.

    FDA. FDA approves pill with sensor that digitally tracks if patients have ingested their medication. 2017 [cited 2017 17 Dec]; Available from:

  53. 53.

    FDA. FDA permits marketing of mobile medical application for substance use disorder. 2017 [cited 2017 17 Dec]; Available from:

  54. 54.

    Oztoprak H, et al. Machine-based classification of ADHD and nonADHD participants using time/frequency features of event-related neuroelectric activity. Clin Neurophysiol. 2017;128(12):2400–10.

    Article  Google Scholar 

  55. 55.

    Emerson RW, Adams C, Nishino T, Hazlett HC, Wolff JJ, Zwaigenbaum L, et al. Functional neuroimaging of high-risk 6-month-old infants predicts a diagnosis of autism at 24 months of age. Sci Transl Med. 2017;9(393).

    Article  Google Scholar 

  56. 56.

    Miner AS, Milstein A, Hancock JT. Talking to machines about personal mental health problems. JAMA. 2017;318(13):1217–8.

    Article  Google Scholar 

  57. 57.

    Rollman BL, Herbeck Belnap B, Abebe KZ, Spring MB, Rotondi AJ, Rothenberger SD, et al. Effectiveness of online collaborative care for treating mood and anxiety disorders in primary careA randomized clinical trial. JAMA Psychiatry. 2018;75(1):56–64.

    Article  Google Scholar 

  58. 58.

    Communication of the ACM. The Real Risks of Artificial Intelligence. 2017 [cited 2017 17 Dec]; Available from:

  59. 59.

    Saeed SA, Johnson TL, Bagga M, Glass O. Training residents in the use of Telepsychiatry: review of the literature and a proposed elective. Psychiatr Q. 2017;88(2):271–83.

    Article  Google Scholar 

  60. 60.

    Sunderji N, Crawford A, Jovanovic M. Telepsychiatry in graduate medical education: a narrative review. Acad Psychiatry. 2015;39(1):55–62.

    Article  Google Scholar 

  61. 61.

    Hilty D, Yellowlees PM, Parrish MB, Chan S. Telepsychiatry. Psychiatric Clinics. 2015;38(3):559–92.

    PubMed  Google Scholar 

  62. 62.

    Pew Research Center. Technology Adoption by Baby Boomers (and everybody else). 2016 [cited 2017 17 Dec]; Available from:

  63. 63.

    Pew Research Center. Social Media Usage: 2005–2015. 2015 [cited 2017 17 Dec]; Available from:

  64. 64.

    Hoge E, Bickham D, Cantor J. Digital media, anxiety, and depression in children. Pediatrics. 2017;140(Suppl 2):S76–80.

    Article  Google Scholar 

  65. 65.

    Lin LY, Sidani JE, Shensa A, Radovic A, Miller E, Colditz JB, et al. Association between social media use and depression among U.S. young adults. Depress Anxiety. 2016;33(4):323–31.

    Article  Google Scholar 

  66. 66.

    Hilty DM, et al. Effects of telepsychiatry on the doctor-patient relationship: communication, satisfaction, and relevant issues. Primary Psychiatry. 2002;9(9):29–34.

    Google Scholar 

  67. 67.

    Profit D, Rohatagi S, Zhao C, Hatch A, Docherty JP, Peters-Strickland TS. Developing a digital medicine system in psychiatry: ingestion detection rate and latency period. J Clin Psychiatry. 2016;77(9):e1095–100.

    Article  Google Scholar 

  68. 68.

    van der Schaar PJ, Dijksman JF, Broekhuizen-de Gast H, Shimizu J, van Lelyveld N, Zou H, et al. A novel ingestible electronic drug delivery and monitoring device. Gastrointest Endosc. 2013;78(3):520–8.

    Article  Google Scholar 

  69. 69.

    Maurer JM, Schellekens RCA, van Rieke HM, Wanke C, Iordanov V, Stellaard F, et al. Gastrointestinal pH and transit time profiling in healthy volunteers using the intelliCap system confirms ileo-colonic release of ColoPulse tablets. PLoS One. 2015;10(7):e0129076.

    Article  Google Scholar 

  70. 70.

    Rotondi AJ, Sinkule J, Haas GL, Spring MB, Litschge CM, Newhill CE, et al. Designing websites for persons with cognitive deficits: design and usability of a psychoeducational intervention for persons with severe mental illness. Psychol Serv. 2007;4(3):202–24.

    Article  Google Scholar 

  71. 71.

    Kopelowicz A, Baker RA, Zhao C, Brewer C, Lawson E, Peters-Strickland T. A multicenter, open-label, pilot study evaluating the functionality of an integrated call center for a digital medicine system to optimize monitoring of adherence to oral aripiprazole in adult patients with serious mental illness. Neuropsychiatr Dis Treat. 2017;13:2641–51.

    Article  Google Scholar 

  72. 72.

    Shafrin J, May SG, Shrestha A, Ruetsch C, Gerlanc NM, Forma F, et al. Access to credible information on schizophrenia patients’ medication adherence by prescribers can change their treatment strategies: evidence from an online survey of providers. Patient Prefer Adherence. 2017;11:1071–81.

    Article  Google Scholar 

  73. 73.

    Rohatagi S, Profit D, Hatch A, Zhao C, Docherty JP, Peters-Strickland TS. Optimization of a digital medicine system in psychiatry. J Clin Psychiatry. 2016;77(9):e1101–7.

    Article  Google Scholar 

  74. 74.

    Borges NJ, Manuel RS, Elam CL, Jones BJ. Comparing millennial and generation X medical students at one medical school. Acad Med. 2006;81(6):571–6.

    Article  Google Scholar 

  75. 75.

    Borges NJ, Manuel RS, Elam CL, Jones BJ. Differences in motives between millennial and generation X medical students. Med Educ. 2010;44(6):570–6.

    Article  Google Scholar 

  76. 76.

    Evans KH, Ozdalga E, Ahuja N. The medical education of generation Y. Acad Psychiatry. 2016;40(2):382–5.

    Article  Google Scholar 

  77. 77.

    Twenge JM. Generational changes and their impact in the classroom: teaching generation me. Med Educ. 2009;43(5):398–405.

    Article  Google Scholar 

  78. 78.

    Pettit RK, McCoy L, Kinney M. What millennial medical students say about flipped learning. Adv Med Educ Pract. 2017;8:487–97.

    Article  Google Scholar 

  79. 79.

    Jordan J, Jalali A, Clarke S, Dyne P, Spector T, Coates W. Asynchronous vs didactic education: it’s too early to throw in the towel on tradition. BMC Med Educ. 2013;13:105.

    Article  Google Scholar 

  80. 80.

    Hughes JK, Kearney P. Impact of an iDevice application on student learning in an occupational therapy kinesiology course. Mhealth. 2017;3:43.

    Article  Google Scholar 

  81. 81.

    Gonzalez NR, Dusick JR, Martin NA. Effects of mobile and digital support for a structured, competency-based curriculum in neurosurgery residency education. Neurosurgery. 2012;71(1):164–72.

    Article  Google Scholar 

  82. 82.

    Englander E, Donnerstein E, Kowalski R, Lin CA, Parti K. Defining cyberbullying. Pediatrics. 2017;140(Suppl 2):S148–51.

    Article  Google Scholar 

  83. 83.

    Hutson E. Cyberbullying in adolescence: a concept analysis. ANS Adv Nurs Sci. 2016;39(1):60–70.

    Article  Google Scholar 

  84. 84.

    Remond JJ, Kern L, Romo L. A cyberbullying study: analysis of cyberbullying, comorbidities and coping mechanisms. Encéphale. 2015;41(4):287–94.

    Article  Google Scholar 

  85. 85.

    Chang FC, Lee CM, Chiu CH, Hsi WY, Huang TF, Pan YC. Relationships among cyberbullying, school bullying, and mental health in Taiwanese adolescents. J Sch Health. 2013;83(6):454–62.

    Article  Google Scholar 

  86. 86.

    Ong R. Confronting online social aggression in Hong Kong: a wake-up call. Int J Law Psychiatry. 2017;54:1–15.

    Article  Google Scholar 

  87. 87.

    Sabin JE, Harland JC. Professional ethics for digital age psychiatry: boundaries, privacy, and communication. Curr Psychiatry Rep. 2017;19(9):55.

    Article  Google Scholar 

  88. 88.

    Hinduja S, Patchin JW. Cultivating youth resilience to prevent bullying and cyberbullying victimization. Child Abuse Negl. 2017;73:51–62.

    Article  Google Scholar 

  89. 89.

    Aboujaoude E, Savage MW, Starcevic V, Salame WO. Cyberbullying: review of an old problem gone viral. J Adolesc Health. 2015;57(1):10–8.

    Article  Google Scholar 

  90. 90.

    Roberto AJ, Eden J, Deiss DM, Savage MW, Ramos-Salazar L. The short-term effects of a cyberbullying prevention intervention for parents of middle school students. Int J Environ Res Public Health. 2017;14(9):1038.

    Article  Google Scholar 

  91. 91.

    Espelage DL, Hong JS. Cyberbullying prevention and intervention efforts: current knowledge and future directions. Can J Psychiatr. 2017;62(6):374–80.

    Article  Google Scholar 

  92. 92.

    IOM (Institute of Medicine) and NRC (National Research Council).In: Building capacity to reduce bullying: workshop summary. 2014: Washington (DC).

Download references

Author information



Corresponding author

Correspondence to Jung Won Kim.

Ethics declarations

Ethical Considerations



Kim, Torous, and Gipson have nothing to disclose. Chan reports grants from the American Psychiatric Association/SAMHSA, personal fees from HealthLinkNow, personal fees from North American Center for Continuing Medical Education, LLC, and personal fees from Guidewell Innovation, outside the submitted work.

Additional information

Shih Yee-Marie Tan Gipson is the Senior Author.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Kim, J.W., Torous, J., Chan, S. et al. Developing a Digitally Informed Curriculum in Psychiatry Education and Clinical Practice. Acad Psychiatry 42, 782–790 (2018).

Download citation