Inappropriate testosterone use and variations in testosterone prescribing patterns exist in the Veterans Health Administration (VHA) despite the presence of clinical guidelines.
We examined system and clinician factors that contribute to patterns of potentially inappropriate testosterone prescribing in VHA.
Qualitative study using a positive deviance approach to understand practice variation in high- and low-testosterone prescribing sites.
Twenty-two interview participants included primary care and specialty clinicians, key opinion leaders, and pharmacists at 3 high- and 3 low-testosterone prescribing sites.
Semi-structured phone interviews were conducted, transcribed, and coded using a priori theoretical constructs and emergent themes. Case studies were developed for each site and a cross-case matrix was created to evaluate variation across high- and low-prescribing sites.
We identified four system-level domains related to variation in testosterone prescribing: organizational structures and processes specific to testosterone prescribing, availability of local guidance on testosterone prescribing, well-defined dissemination process for local testosterone polices, and engagement in best practices related to testosterone prescribing. Two clinician-level domains were also identified, specifically, structured initial testosterone prescribing process and specified follow-up testosterone prescribing process. High- and low-testosterone prescribing sites systematically varied in the four system-level domains, while the clinician-level domains looked similar across all sites. The third high-prescribing site was unusual in that it exhibited the four domains similar to the 3 low-prescribing sites at the time of our visit. This site had greatly reduced its prescribing of testosterone in the interim.
Findings suggest that local organizational factors play an important role in influencing prescribing. Sites have the potential to transform their utilization patterns by providing access to specialty care expertise, an electronic health record-based system to facilitate guideline-concordant prescribing, well-defined dissemination processes for information, guidance from multiple sources, and clarity regarding best practices for prescribing.
Testosterone replacement therapy has been approved by the US Food and Drug Administration (FDA) to treat male hypogonadism.1 However, the past decade witnessed dramatic increased testosterone use in patients who do not have hypogonadism2,3 implying inappropriate testosterone prescribing. Several contributors to inappropriate prescribing include an increase in testosterone level testing, nonadherence to guidelines at initiation and monitoring of testosterone therapy, and prescribing of this therapy for non-indicated conditions.3 Other contributors may include aggressive marketing efforts, establishment of dedicated testosterone prescribing clinics, availability of new, easier modalities for testosterone administration, and ambiguous clinical recommendations on the appropriateness of prescribing testosterone for age-related decline of testosterone levels.3,4 Motivated by safety concerns on the potential link between testosterone use and cardiovascular risk,5 in 2015, the FDA mandated new labeling of testosterone products and approved indications for its use, thereby changing prescribing by making the treatment of age-related and idiopathic hypogonadism “off-label.”6 This effort by the FDA, along with articles in the lay press,7 may have contributed to a decline in testosterone prescribing.3
Several guidelines exist on the diagnostic ascertainment and management of hypogonadism;1,8,9 however, wide-scale variability of clinical practice persists.10 Though we would expect higher guideline-concordant testosterone prescribing in Veterans Health Administration (VHA) due to presence of a national formulary and an organized pharmacy, both VHA and non-VHA studies have documented similar testosterone prescribing patterns with insufficient laboratory workup before receipt of testosterone and high levels of use for unapproved indications.3 We must first understand the causes and context of this variation to successfully address suboptimal prescribing.11
Previous studies found the decision to prescribe medication results from complex interactions between patient-, clinician-, and system-level factors (Fig. 1).12 Following this model, when a patient reports symptoms and expresses expectations for (testosterone) therapy, the decision to prescribe depends on the clinician’s specialty/judgment, formal policies, and informal practice patterns at the site. Understanding this context will enable the design of interventions to improve testosterone prescribing.
We sought to examine system and clinician factors that contribute to the patterns of testosterone prescribing in VHA. Based on our previous finding that the majority of prescriptions were written for unapproved, off-label indications,2 we argue that higher testosterone prescribing is most often inappropriate and guideline-discordant. This assumption of less testosterone prescribing equating to more appropriate prescribing finds support in the antibiotic prescribing field.13,14 In fact, low ideal antibiotic prescribing proportions have been recommended to serve as benchmarks for appropriate prescribing.14 Hence, we sought to examine the relationship between appropriate prescribing practices and the level of testosterone prescribing. We used a positive deviance approach15 by qualitatively examining sites that were high and low outliers on testosterone prescribing, to understand the context within which practice variation occurs.
We conducted a qualitative study at six VHA medical centers. The study was approved by the Institutional Review Board of the Bedford VHA Medical Center.
We used data from the VHA Corporate Data Warehouse (CDW) to profile all VHA sites on testosterone prescribing in fiscal year (FY)14. Testosterone utilization across the entire VHA system in FY14 was 2.30% of all males. We ranked all 130 VHA sites on their rates of testosterone prescribing in FY14 from highest to lowest and identified three high and three low testosterone prescribing sites from among the 10 highest and 10 lowest sites.
Participants included primary care and specialty clinicians, pharmacists, and clinical leaders. At each site, we first identified clinicians who had written 50 or more outpatient testosterone prescriptions in FY14. Of these, we selected the clinicians with the most and least prescriptions at the site. Additionally, we identified key opinion leaders such as the chiefs of pharmacy, primary care, endocrinology, and urology (when present) to understand local policies and practices that inform the use of testosterone. We requested participation via email; those who expressed interest were contacted by phone.
Tailored interview guides (Online Appendix A and B) addressed two factors of the conceptual model (system-level and clinician-level) (Fig. 1).12 The third factor (patient-level) was not the focus for this specific study. Semi-structured phone interviews were conducted by trained qualitative interviewers (GJ, RE, AS) from January to June 2016. Based on the conceptual model, interviews focused on clinician-specific initial prescribing and follow-up processes, site-specific testosterone therapy setting, dissemination process, concerns and challenges regarding testosterone therapy, existing policies and guidance, and individual opinions regarding prescribing testosterone therapy. Participants were not informed of their status as low or high outlier sites, because of the potential to influence responses.15
Qualitative analysis was conducted by a four-person qualitative team comprised of experts in qualitative research methods (BB and RE) and prescribing (GJ and AS). A codebook was developed based on system, clinician, and patient factors in the conceptual model (Fig. 1) in addition to emergent themes. Coders (GJ, RE, and AS) reached consensus by co-coding five interviews in an iterative process. Once consensus regarding coding categories was achieved, each interview was coded by two coders, using NVivo 10 software. Analysts (GJ and RE) each independently reviewed coding from three sites. Data was summarized in a case study for each site and a cross-case matrix16 was created to determine variation across high- and low-prescribing sites.
The 3 high- and 3 low-testosterone prescribing sites were located in six states and all four regions of the USA. High-prescribing sites prescribed testosterone to > 4% of male patients, while low-prescribing sites prescribed to < 1%. Details about the sites’ testosterone utilization rates and participants are provided in Table 1.
We identified four system-level and two clinician-level domains related variation in testosterone prescribing. These four system-level domains were (1) organizational structures and processes specific to testosterone prescribing, (2) availability of local guidance on testosterone prescribing, (3) well-defined dissemination process for local testosterone polices, and (4) engagement in best practices related to testosterone prescribing. Two clinician-level domains were (5) structured initial testosterone prescribing process and (6) specified follow-up testosterone prescribing process. Table 2 provides definitions for each of these identified domains and the domain’s relation to the conceptual model (Fig. 1).
High- and low-prescribing sites systematically differed on the 4 system-level domains, whereas domains 5 and 6 (clinician-level domains) were largely similar across all sites. At all sites, most testosterone prescriptions were initiated by patient request and clinicians varied in their adherence to guideline concordant prescribing. Among the high-prescribing group, one site was unusual in that it exhibited domains similar to the 3 low-prescribing sites; we will discuss this site in greater detail below. We first report results on the four system-level domains that reliably differentiated between low and high sites, followed by a case study of the anomalous high-prescribing site.
Domain #1 (System Factor): Organizational Structures and Processes Specific to Testosterone Prescribing
Low-prescribing sites had specified organizational structures and processes for (1) who prescribes testosterone and (2) use of an electronic system to influence prescribing. These were not present in the high-prescribing sites.
Who Prescribes—Primary Care or Specialty Care?
Though all study sites had some kind of referral or consult process to a specialist, low-prescribing sites referred most or all patients to a specialist prior to receiving testosterone, whether by policy or simply by pattern. One leader from a low-prescribing site reported that testosterone prescribing was low at his site due to the culture of sending all patients to the urologist:
It does sort of support my theory that we’ve always been slightly conservative, but I think it’s because our culture here. We’ve had most of these patients going to the urology clinic rather than through primary care providers. (Chief of Pharmacy, Site A)
At high-prescribing sites, primary care physicians (PCPs) generally prescribed testosterone without specialist input. Clinicians at these sites also reported difficulty accessing specialty care from endocrine or urology, two specialties that can help ensure appropriate testosterone prescriptions. Sites often had a backlog of endocrine consults (which were often outsourced to another VHA medical center) or had no endocrinologist or urologist onsite.
Use of an Electronic System to Influence Prescribing
At all three low-prescribing sites, some form of electronic system was present to influence the testosterone prescribing process. For example, all low-prescribing sites built a restricted drug request (RDR) or reminder into the electronic health record (EHR) that helped ensure alignment with guidelines. One participant described the RDR:
We have a restricted drug request which is required for testosterone. And I don’t know if that is a structure that other VAs have or not, but it is a long and detailed template that you have to go through to submit a request for testosterone and then the pharmacy will take a few days and decide whether or not to issue it. And that has to be repeated every six months. (Chief of Primary Care, Site C)
The existence of this request form discouraged testosterone prescriptions (by requiring extra effort), but also encouraged clinicians to carefully consider whether a patient has met criteria to receive testosterone before prescribing. This feature was not present at two of the three high-prescribing study sites.
Domain #2 (System Factor): Availability of Local Guidance on Testosterone Prescribing
Low-prescribing sites offered a variety of different forms of guidance to help improve testosterone prescribing. Indeed, one informant at a low-prescribing site stated that providing such guidance was a form of clinician education. A respondent at a low-prescribing site described existing guidance:
We made a drug file link, so every time the providers went to order testosterone, there was a blue link that says ‘display restrictions and guidelines’ and when they clicked on that, that brought them to the screen that said, you know, ‘You need to do the morning levels.’ (Chief of Pharmacy, Site A)
Low-prescribing sites offered a wider variety of guidance formats for testosterone prescribing, including circulation of pharmacy benefits management (PBM) criteria for use (CFU), regional guidelines, and safety alerts from the local pharmacy and therapeutics (P&T) committee to clinicians. Additionally, they disseminated relevant literature and guidance from the endocrine service. At one of these low-prescribing sites, the RDR form used by clinicians for testosterone prescribing also served as guidance. At two of the high-prescribing sites, guidance was limited as was clinician awareness; informants at one site were not even aware of the existence of VHA guidance on testosterone use.
Domain #3 (System Factor): Well-Defined Dissemination Process for Local Testosterone Policies
The low-prescribing sites had well-defined committee structures, processes, and resources in place for dissemination of polices regarding testosterone to medical staff. Importantly, these processes occurred at both local and regional levels. At one low-prescribing site, the P&T committee shared information with pharmacists in monthly meetings. At the other two low-prescribing sites, dissemination involved circulation of materials produced by VHA’s national PBM service through emails and monthly newsletters. At one low-prescribing site, the regional pharmacy service also pursued in-person meetings with clinicians to discuss the CFU:
…Our drug information specialist and three different committees…will send out emails specifically to the providers and tell them this is how you should be doing this. A lot of times even the folks in the [VA regional pharmacy service] will make face-to-face appointments with the providers to sit down with them and just go over the criteria for use. The other thing is that these criteria for use are available for everyone at the site to pull up and review. (Clinical Pharmacist, Site C)
In contrast, there was less evidence of a well-defined dissemination structure at two of the three high-prescribing sites. One high-prescribing site clinician stated:
We just follow the routine general practice only. I don’t think we have any actual VA guidelines on testosterone therapy. I don’t think there’s any brochures or literature here on the VA policy. (Physician, Site E)
At a third high-prescribing site, dissemination of policies was limited, relying solely on local leaders directly communicating with clinicians. In fact, several leaders at this site told us they could use help from academic detailers to help educate clinicians on testosterone guidelines, because they were often too busy to do so.
Domain #4 (System Factor): Engagement in Best Practices Related to Testosterone Prescribing
Participants at low- and high-prescribing sites also differed in what they believed to be best practices for testosterone prescribing. Low-prescribing sites felt best practices should include giving clinicians easily accessible information, which should be integrated into the EHR to enable clinicians to make informed prescribing decisions in real time. Informants also emphasized the need to make sure guidelines or best practices are consistent throughout VHA. One clinician described this need for consistency:
I think coming up with a best practices or clinical practice guideline that would be consistent throughout the VA would be the best situation, and that would come from the national level. If the Veteran travels or switches from one VA to another and you’ve got different VA’s doing different things, it just gets confusing and then the patient gets confused. And I just think it’s best to come from a national perspective, so everybody’s on the same page. (Clinical Pharmacist, Site B)
Patient education through materials shared by the clinician with the patient was also identified as a best practice. Informants at low-prescribing sites also mentioned consistent adherence with testing requirements prior to prescribing testosterone, such as documenting two low morning testosterone levels, as important.
The knowledge of best practices at high-prescribing sites was limited, and beliefs diverged from both VHA and non-VHA clinical practice guidelines. Informants at two high-prescribing sites could not name any important best practices they use to inform testosterone prescribing. At one high-prescribing site, the endocrinologist stated that he relied more on laboratory-measured levels of testosterone than on symptoms when deciding who should receive testosterone (which is contrary to guideline recommendations). A PCP at the same site mentioned everyone over age 50 at the site received routine testosterone level testing, regardless of presence or absence of relevant symptoms. This practice is not recommended by any guideline and would increase the number of patients for whom testosterone therapy would be considered.
High-Prescribing Outlier Site: a Case Study
One site was unique in that it exhibited many of the strategies that contributed to low prescribing at other sites, but the level of prescribing at this site was in fact high. We found this site had engaged in significant quality improvement efforts regarding testosterone prescribing in the period between our initial ranking in FY14 and interviews conducted in FY16. This high-prescribing outlier was one of the highest testosterone utilizing facilities in the VHA in FY14, with a testosterone prescribing rate of 4.1%, approximately twice the national VHA rate. Stakeholders at this site identified inappropriate use of testosterone as a priority in FY16. As part of this initiative, this high-prescribing outlier established an endocrine task force to address certain challenges, including providing data on inappropriate testosterone use and decreasing inappropriate use. To achieve these goals, this site employed a variety of strategies, including developing their own testosterone use criteria; creating a medication safety dashboard (a population management tool) focused on testosterone prescribing; extensive academic detailing for clinicians on testosterone; and provision of educational pamphlets for patients about risks and benefits of testosterone.
This high-prescribing site was similar to the 3 low sites in the following ways:
PBM CFU: A regional taskforce was established to review and improve testosterone prescribing practices. The taskforce completed chart reviews to compare care with national PBM CFU recommendations, and distributed the CFU locally. One participant described the taskforce:
So, we have an endocrine taskforce that were staffing and providing data that is looking at, ‘What is the baseline inappropriate use of testosterone? How can we improve adherence to our criteria for use? And how can we go about actually decreasing inappropriate use?’ …there was basically a disbelief that we had a problem with testosterone, among the endocrinologists. So the first thing we did was we did a random chart review of fifty patients who were on chronic testosterone therapy at each of our medical centers. That was basically checked against the criteria for use that we had developed previously. (Regional-level Pharmacy Executive, Site D)
Academic detailing: Additionally, academic detailers provided one-on-one sessions with clinicians at this site to educate them about criteria for prescribing testosterone and risks/benefits associated with this therapy. For example:
Every month we meet with pharmacy. All the providers meet with pharmacy. They usually guide the agenda and talk about sort of some of their metrics, so testosterone is becoming one of those metrics, and so they have given us information that they are sharing with patients, so if the patient is on testosterone, they mail them an informational flyer. It has the information and then they bring the subject up with their provider at their next meeting. (Associate Chief Ambulatory Care, Site D)
Testosterone dashboard: Similar to the low-prescribing sites, this site had a population management tool in the form of a dashboard linked to the Computerized Patient Record System (CPRS) to assess patients based on guideline criteria:
[The Testosterone Dashboard is] a separate computer program. You can access it through CPRS. And all of our clinicians currently have access to it. So, it’s a tool that our region has developed to help us with population management by identifying various population groups, based on demographic factors that we decided or risk factors that we decided, to help us kind of intervene and prevent issues before they are happening. (Clinical Pharmacist, Site D)
Thus, this third high-prescribing site became aware of their status and made testosterone prescribing a high priority. In fact, this site did decrease the absolute level of testosterone prescribing by approximately 2% between FY14 and FY17 based on their improvement efforts. However, it should be noted that even in FY17, the testosterone prevalence rate at this site (2.06%) remained higher than the national rate of 1.45%, likely reflecting the fact that many patients cannot be taken off a medication easily once they have begun it.
Understanding the context within which testosterone prescribing occurs is key to planning successful interventions to improve guideline-concordant prescribing.15 This issue of inappropriate testosterone prescribing is especially concerning in males with an unapproved indication due to the associated cardiovascular risk. Previous VHA-based studies of prescribing patterns have similarly revealed organizational and clinician-level factors contributing to inappropriate or inadequate prescribing.17,18,19 In this study, we found that low-prescribing sites shared some common features, including easier access to specialty care expertise, existence of an EHR-based system to facilitate guideline-concordant prescribing, well-defined dissemination processes for information, availability of guidance from multiple sources, and clarity regarding what constitutes best practices for prescribing. In contrast, neither initial decision-making nor specified follow-up processes distinguished the level of prescribing at a site. System-level domains in our conceptual model, specifically practice setting, local policies, and engagement, differed between low- and high-prescribing sites. The clinician-level domains, in contrast, showed little variation between high- and low-prescribing sites.
We also observed that one of our high-prescribing sites had undertaken change processes between when their prescribing rate was measured (FY14) and when we interviewed them (FY16). In the interim, they had adopted practices that were extremely similar to low-prescribing sites. Their adoption of a multi-level strategy with separate interventions at the patient (educational brochure), clinician (academic detailing), and system-level (population management tool, development of own CFU) bespeak considerable attention to improving the quality of testosterone prescribing. In fact, this site has succeeded in reducing its testosterone prescribing rate by almost half within 2 years, suggesting these efforts have had considerable impact.
One of our implicit assumption was that less prescribing of testosterone is usually associated with a lower proportion of inappropriate prescriptions. This is supported by our previous study2 and by analogy with the literature about antibiotic prescribing.13,14 However, the relationship between absolute prescribing levels and patterns of appropriateness at the site-level may not always be completely predictable or consistent. In another study, we also found higher rates of prescribing were associated with higher rates of appropriate pretreatment diagnostic evaluation.20 Therefore, while we continue to believe that lower prescribing at the site-level generally equates with appropriate prescribing to those who do receive testosterone, this may not be without exception.
Building upon the quantitative findings from our recent studies,2,20 this qualitative study seeks to further identify targets of intervention to improve prescribing of testosterone in the VHA. Our findings suggest that more effective dissemination of guidance in the form of guideline recommendations to clinicians could improve evidence-based prescribing of testosterone, quality of care, patient outcomes, and safety.21 The role of the system in disseminating and implementing these guidelines and making them easily accessible to the clinician to inform the prescribing decision-making process is also likely important.22 Previous evidence suggests successful implementation of guidelines resulted from interventions not only targeted at changing the clinician behavior, but also working towards involving patients and appropriate system stakeholders to develop adequate strategies for guideline dissemination and implementation.23
Our finding of easier access to specialist expertise at low-prescribing sites suggests the association of this care with more appropriate prescribing. In the literature, specialists have been shown to be generally knowledgeable about their area of expertise. Their practice tends to comply with treatment guidelines and they tend to achieve better outcomes in these areas of practice than generalists.24 In a previous study, we reported that endocrinologists were twice as likely to obtain an appropriate workup before initiating testosterone, compared to PCPs.20 Since structured initial decision-making testosterone prescribing process (domain 5) did not vary between low- and high-testosterone prescribing sites, we speculate that endocrinologists are potentially exerting their effect through other means, e.g., local CFU, academic detailing, or contributions to the design of electronic clinical reminders. Taken together with the present study, these findings suggest that providing better access to specialty care could potentially improve access to evidence-based testosterone prescribing. The use of national electronic consults25,26,27 and specialty training programs such as Specialty Care Access Network-Extension of Community Healthcare Outcomes (SCAN-ECHO)28,29 in the VHA to bring specialist expertise within reach has the potential to contribute to improved quality of care.
To our knowledge, this is the first study to examine the context of testosterone prescribing in an integrated healthcare system. In addition, we used a positive deviance design, which provided additional insight on site-level factors associated with variant practice patterns. This study does have some limitations. First, we conducted telephone instead of in-person interviews, which may not always be optimal because of absence of visual cues and compromised rapport.30 However, the detailed interview data and duration of the interviews imply this modality worked well for our study. Second, we were not blinded to the status (high- or low-prescribing) of our sites while conducting interviews or during analyses of data. However, we purposefully tried to record what was working in low-prescribing sites and what was lacking in high-prescribing sites.16 Third, at two of the high-prescribing sites, we were not able to recruit more than one leader to be interviewed for our study. This lack of participation may reflect the minimal organization of testosterone care that we found. Fourth, VHA may not be representative of other healthcare systems in certain respects due to presence of an organized pharmacy service, a centralized national formulary, and a highly functional electronic medical record system. Fifth, we acknowledge that creating electronic systems to influence prescribing, such as RDRs, may have the potential to be burdensome to clinicians, which could contribute to unintended consequences, such as clinician failure to prescribe testosterone even when appropriate. Finally, though we recognize the critical role of the patient as an essential participant in prescribing decision-making, this study does not include the patient perspective on testosterone prescribing. However, we are examining patient perspectives in a separate study.
Findings from this study suggest that local organizational factors play an important role in influencing prescribing, and offer the opportunity to focus quality improvement efforts. Sites have the potential to transform themselves, and their utilization patterns, by focusing on the four domains we identified as distinguishing features between high- and low-prescribing sites. The use of these system-level factors to change medication prescribing can also be extrapolated to improving other prescribing behavior, including safer prescribing of high-risk drugs and facilitating deprescribing of unnecessary and/or harmful medications. Lessons learned from this study will serve as an exemplar for any site aiming to improve its testosterone prescribing practices and prescribing in general beyond testosterone.11
Bhasin S, Brito JP, Cunningham GR, et al. Testosterone Therapy in Men With Hypogonadism: An Endocrine Society Clinical Practice Guideline. J Clin Endocrinol Metab., 2018; 103(5): 1715–44.
Jasuja GK, Bhasin S, Reisman JI, et al. Who gets Testosterone? Patient Characteristics Associated with Testosterone Prescribing in the Veteran Affairs System: A Cross-Sectional Study. J Gen Intern Med. 2017;32:304.
Jasuja GK, Bhasin S, Rose AJ. Patterns of testosterone prescription overuse. Curr Opin Endocrinol Diabetes Obes. 2017. https://doi.org/10.1097/MED.0000000000000336.
Mascarenhas A, Khan S, Sayal R, et al. Factors that may be influencing the rise in prescription testosterone replacement therapy in adult men: a qualitative study. Aging Male 2016; 19(2):90–5.
Editorial. Testosterone Therapy: Has Overuse Undermined Use? Lancet Diabetes Endocrinol. 2018;6(3): 157.
Desroches B, Kohn TP, Welliver C, Pastuszak AW. Testosterone therapy in the new era of Food and Drug Administration oversight. Transl Androl Urol. 2016;5(2):207–12.
Tavernise S. FDA Panel Backs Limits on Testosterone Drugs. New York: The New York Times Company. Available at: http://nyti.ms/1uU2VZ1. Accessed: January 1, 2019.
Lunenfeld B, Mskhalaya G, Zitzmann M, et al. Recommendations on the diagnosis, treatment and monitoring of hypogonadism in men. Aging Male. 2015;18(1):5–15. doi:https://doi.org/10.3109/13685538.2015.1004049.
Petak SM, Nankin HR, Spark RF, et al. American Association of Clinical Endocrinologists Medical Guidelines for clinical practice for the evaluation and treatment of hypogonadism in adult male patients – 2002 update. Endocr Pract. 2002;8:440–56.
Grossmann M, Anawalt BD, Wu FC. Clinical practice patterns in the assessment and management of low testosterone in men: an international survey of endocrinologists. Clin Endocrinol (Oxf). 2015;82(2):234–41.
Rose AJ, McCullough MB, Jasuja GK. A three-step health services research approach to improve prescribing. Healthc (Amst). 2017.
Aagaard EM, Gonzales R, Camargo CA Jr, et al. Physician champions are key to improving antibiotic prescribing quality. Jt Comm J Qual Patient Saf. 2010;36(3):109–16.
Pouwels KB, Dolk FCK, Smith DRM, Robotham JV, Smieszek T. Actual versus ‘ideal’ antibiotic prescribing for common conditions in English primary care. J Antimicrob Chemother, 2018; 73(2), 19–26.
Smith DRM, Dolk FCK, Pouwels KB, Christie M, Robotham JV, Smieszek T. Defining the appropriateness and inappropriateness of antibiotic prescribing in primary care. J Antimicrob Chemother. 2018; 73(2), ii11–ii18.
Rose AJ, McCullough MB. A Practical Guide to Using the Positive Deviance Method in Health Services Research. Health Serv Res. 2017;52(3):1207–1222.
Miles MB, Huberman AM. Qualitative data analysis: An expanded sourcebook (2nd ed.). Thousand Oaks, California: Sage Publications; 1994.
Livorsi D, Comer AR, Matthias MS, Perencevich EN, Bair MJ. Barriers to guideline-concordant antibiotic use among inpatient physicians: A case vignette qualitative study. J Hosp Med. 2015;11(3):174–80.
Livorsi D, Comer A, Matthias MS, Perencevich EN, Bair MJ. Factors Influencing Antibiotic-Prescribing Decisions Among Inpatient Physicians: A Qualitative Investigation. Infect Control Hosp Epidemiol. 2015;36(9):1065–72.
Goren J, Rose AJ, Engle RL, et al. Organizational Characteristics of Veterans Affairs Clinics With High and Low Utilization of Clozapine. Psychiatr Serv. 2016;67(11):1189–1196.
Jasuja GK, Bhasin S, Rose AJ, et al. Provider and Site-Level Determinants of Testosterone Prescribing in the Veterans Healthcare System. J Clin Endocrinol Metab. 2017; 102(9):3226–3233.
Woolf SH, Grol R, Hutchinson A, Eccles M, Grimshaw J. Potential benefits, limitations, and harms of clinical guidelines. BMJ. 1999;318(7182):527–530.
Fischer F, Lange K, Klose K, Greiner W, Kraemer A. Barriers and Strategies in Guideline Implementation—A Scoping Review. Parthasarathy S, ed. Healthcare. 2016;4(3):36. doi:https://doi.org/10.3390/healthcare4030036.
Forsner T, Hansson J, Brommels M, Wistedt AÅ, Forsell Y. Implementing clinical guidelines in psychiatry: a qualitative study of perceived facilitators and barriers. BMC Psychiatry. 2010;10:8. doi:https://doi.org/10.1186/1471-244X-10-8.
Harrold LR, Field TS, Gurwitz JH. Knowledge, Patterns of Care, and Outcomes of Care for Generalists and Specialists. J Gen Intern Med. 1999;14(8):499–511. doi:https://doi.org/10.1046/j.1525-1497.1999.08168.x.
Vimalananda VG, Gouri G, Seraj SM, et al. Electronic consultations (e-consults) to improve access to specialty care: a systematic review and narrative synthesis. J Telemed Telecare. 2015;21(6):323–330.
Gupte G, Vimalananda V, Simon SR,et al. Disruptive Innovation: Implementation of Electronic Consultations in a Veterans Affairs Health Care System. JMIR Med Inform. 2016;4(1):e6.
Strymish J, Gupte G, Afable MK, et al. Electronic Consultations (E-consults): Advancing Infectious Disease Care in a Large Veterans Affairs Healthcare System. Clin Infect Dis. 2017;64(8):1123–1125.
Chokshi DA, Scheteman G, Agarwal M. Patient-centered innovation: The VA approach. Healthcare. 2013;1(3):72–75.
Kirsh S, Grace S, Sales A, Jain R. Access to outpatient specialty care: Solutions from an integrated health care system. Am J Med Qual. 2014;30(1):88–90.
Novick G. Is There a Bias Against Telephone Interviews In Qualitative Research? Research in Nursing & Health. 2008 31(4), 391–398.
The research reported/outlined here was supported by the Department of Veterans Affairs, Veterans Health Administration, Health Services Research and Development Service. Dr. Jasuja is a VA HSR&D Career Development awardee at the Bedford VA (CDA 13-265). The views expressed in this article are those of the author(s) and do not necessarily represent the views of the Department of Veterans Affairs. The funding sources had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and the decision to submit the manuscript for publication.
The study was approved by the Institutional Review Board of the Bedford VHA Medical Center.
Conflict of Interest
The authors declare that they do not have a conflict of interest.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Electronic Supplementary Material
About this article
Cite this article
Jasuja, G.K., Engle, R.L., Skolnik, A. et al. Understanding the Context of High- and Low-Testosterone Prescribing Facilities in the Veterans Health Administration (VHA): a Qualitative Study. J GEN INTERN MED 34, 2467–2474 (2019). https://doi.org/10.1007/s11606-019-05270-6