INTRODUCTION

Online review platforms have the ability to amplify an individual’s voice. The emergence of these platforms provides an open and public forum for patients to express their opinions which may help redefine how health systems approach the patient experience.

With over 140 million reviews, Yelp! (www.yelp.com), the online review platform, continues to grow in scale and popularity. Patients have begun to share their opinions and ratings with over 8.4 million healthcare related reviews.1, 2 Though online reviews of individual providers may be low yield,3 these reviews may expose blind spots not traditionally considered by hospitals. It is the observability and scale of these online reviews that provides an opportunity for both innovation and promise.4

Our objective was to analyze hospital reviews posted on Yelp through the application of machine learning and natural language processing. We hypothesized there would be common language correlated with negative and positive reviews.

METHODS

This was a retrospective analysis of US hospital reviews on Yelp over 12 years (May 2005–March 2017). Hospital reviews were identified using the facilities reporting data to the American Hospital Association. Each review included narrative text, date, and star rating. Reviews were of facilities and not individual providers. This study was considered exempt by the University of Pennsylvania Institutional Review Board. A machine learning tool, Differential Language Analysis (DLA) was used to identify word frequency and correlation with 1-star (negative) or 5-star (positive) reviews. DLA provides a method of open-vocabulary analysis to determine correlation, using frequency and Pearson’s correlation, among individual words and star rating.

RESULTS

Over 12 years, there were 51,376 reviews from 1566 hospitals. 41.9% were 1-star and 31.6% were 5-stars (mean 2.84 stars). “Told” was the single most correlated word among 1-star reviews, present in 9578 reviews (18.6%) with an average of 1.79 stars. Figure 1 displays a word cloud of all words correlated with 1-star reviews with color representing frequency and size representing correlation. An example of narrative text, “I constantly told them that none of that was true and the nurse there wouldn’t believe me.” The words “great” and “friendly” were the words most correlated with 5-star reviews (Fig. 2). “Friendly” was present in 5594 reviews (10.9%) with an average of 4.02 stars. An example of narrative text, “the doctors, nurses and x-ray technician who helped me out were all so cool and friendly. It really restored my faith in humanity after I got hit on my bike.”

Fig. 1
figure 1

Words most correlated with negative (1-star) hospital reviews on Yelp. Using Differential Language Analysis (DLA), hospital reviews posted on Yelp were analyzed. The color gradient displays weak or strong correlations to the negative reviews. Gray words have weak correlation with a negative review and darker words, as displayed in gradient key, indicate higher correlation. Text size indicates frequency of word use within reviews—larger text size represents a word used more frequently

Full size image
Fig. 2
figure 2

Words most correlated with positive (5-star) hospital reviews on Yelp. Using Differential Language Analysis (DLA), hospital reviews posted on Yelp were analyzed. The color gradient displays weak or strong correlations to the negative reviews. Gray words have weak correlation with a positive review and darker words, as displayed in gradient key, indicate higher correlation. Text size indicates frequency of word use within reviews—larger text size represents a word used more frequently

Full size image

DISCUSSION

This study has two key findings related to online hospital reviews and their content. First, the single word most correlated to negative reviews was “told.” Second, the most correlated words with positive reviews were “great” and “friendly.” Machine learning and natural language processing offer an approach to analyzing the large amount of narrative text within reviews and begin to shed light on issues around communication and interactions between patients and providers.

The growth and expansion of online narrative reviews provides a rich and large data source. There are clearly marked differences and limitations between the standardized national surveys and unfiltered online reviews. Online reviews are not validated and individuals may have external motivations to intentionally drive ratings of a hospital or organization. Furthermore, these reviews are applied at the hospital level and thus cannot be generalized to the individual provider.

Distilling the vast amount of narrative responses from millions of reviews would prove to be onerous and impractical. Though individuals are reviewing and commenting on their unique encounters, the collective themes within a region, system, or specialty may highlight shared experiences.

As healthcare transforms, there has been a focus on becoming more patient-centered. Patient-centered care requires providers to effectively communicate with their patients: to ask, to listen, and to explain.5 Providers must be cognizant of the words and language they use.6 Teaching the ability to communicate through caring language is crucial for healthcare leaders and educators.

Patients have already thought differently about how to make their voices heard by taking to online platforms to review their experiences. In an effort to become patient-centered, providers must begin to think differently about how to listen and communicate. Capturing digital patient generated data through methods such as machine learning can provide an approach to improving the patient experience.