We conducted a retrospective, longitudinal cohort study of 4,235,380 patients who received VA care (had at least one visit annually) between January 2011 and January 2015. This time window was selected to best examine the immediate impact of VA Notes implementation, early-on after becoming nationally available. Among this cohort, we examined patients who used the portal (N = 882,575), comparing patients who accessed notes through the portal (notes users) and those who did not (non-users).
We examined the impact of VA Notes on communication with providers via secure messaging and primary care utilization, comparing differences between notes users and non-users before and after January 2013. We further explored the readability of clinical notes before and after January 2013. This study was reviewed and approved by the Edith Nourse Rogers Memorial Veterans Hospital and the University of Massachusetts Medical School Institutional Review Boards.
Data Sources and Measures
Data for the variables of interest (demographic characteristics, use of the portal, use of healthcare services, text of clinical notes, frequency of viewing notes and secure messages, number of primary care visits) were extracted from the VA corporate data warehouse for the time period from January 2011 to January 2015. As noted above, notes first became available to all patients in January 2013, and data on patient viewing of notes became available in July 2013. The “Pre-notes” (i.e., before implementation) time period was defined as July 2011–January 2013 and the “Post-notes” (i.e., after implementation) time period was defined as July 2013–January 2015.
To address the first research question, we examined demographic and clinical characteristics of all VA patients and compared them with patients with portal accounts who did and did not access notes. Accessing notes was defined as a patient viewing or downloading their clinical notes online using the portal between July 2013 and January 2015. Demographic characteristics included age, gender, race, a marker of financial vulnerability (defined as eligibility for free care based on an annual VA financial assessment), marital status, distance from the nearest facility, residing in an urban or rural setting, and smoking status. Clinical characteristics included indicators for high priority chronic diseases in VA. We examined differences in characteristics between notes users and non-users using chi-square tests.
The second research question explored the relationship between patients accessing notes, and utilization of secure messaging and clinical services. Utilization was measured by the counts of secure messages and the number of primary care visits in time periods of 6 months and 1 year, respectively, between June 2011 and January 2015. We report the mean and median of these counts in 6-month intervals between these dates, with the difference between notes users and non-users for each time period. We conducted interrupted time series analyses, implemented using a segmented regression method, comparing the change in utilization of secure messaging and primary care provider appointments among those who did or did not access notes. The model was implemented using a generalized estimating equation appropriate for repeated measures (annual rates per year) within individuals, and using an auto-regressive correlation structure. Variables were included in the model to represent group (notes users and non-users), study time, pre-post implementation, and interaction terms to test change in slope within groups, as well as the difference of differences in slopes (group by time). We chose not to adjust for number of chronic conditions because we conceptualized them as part of the causal pathway driving the association between access to notes and increased utilization of services. In this large sample, we recognize that even small differences in slope of change may be significant. Thus, in addition to reporting statistical significance, we identified prior work that demonstrated a change in visits, and use a change of 0.7 clinic visits per year as a meaningful cutoff.30
The final research question examined the readability of notes before and after they became available to patients. To accomplish this, a purposeful sample of VA primary care providers (PCPs) with the highest rates of patients accessing their notes under their care was identified (top 1% nationally). Our rationale for targeting this sample was that clinicians with a higher proportion of patients who accessed their notes may be more likely to alter their documentation behaviors. Among the 40 PCPs who had the highest percentage of portal users in their panel, we randomly selected 100 patients who had accessed notes. From each of these patients, we pulled 3 notes pre-implementation and 3 notes post-implementation, and examined them for readability. Readability was assessed using universally accepted scales for evaluating readability of medical information:31 the Flesch reading ease scores (FRES) and the Flesch-Kincaid grade level (FKGL).32,33 The FRES quantifies how easy text is to read on a scale from 0 to 100 with a higher score indicating easier to read text and the FKGL estimates the grade level of text by assessing word count, words per sentence, and average number of syllables per word. The number and frequency of abbreviations were also assessed. These metrics were assessed for the entire note, and its subcomponents, including the history of present illness, the assessment, and the plan. These subcomponents were selected based on relevance to chronic disease management and because they are least likely to be influenced by copy-and-paste templates, unlike physical exam notes. Readability metrics were calculated using Microsoft Office Word 2016 built-in calculator. Changes in readability statistics from pre- to post-VA Notes implementation were determined using paired t tests. Analyses were performed in STATA software version 13.1 (StataCorp, College Station, TX).