Original Article

Academic Psychiatry

, Volume 35, Issue 1, pp 27-34

First online:

Development and Initial Testing of a Structured Clinical Observation Tool to Assess Pharmacotherapy Competence

  • John Q. YoungAffiliated withDepartment of Psychiatry, UCSF School of Medicine Email author 
  • , Sandra LieuAffiliated withDepartment of Psychiatry, UCSF School of Medicine
  • , Patricia O’SullivanAffiliated withOffice of Medical Education, UCSF School of Medicine
  • , Lowell TongAffiliated withDepartment of Psychiatry, UCSF School of Medicine

Rent the article at a discount

Rent now

* Final gross prices may vary according to local VAT.

Get Access



The authors developed and tested the feasibility and utility of a new direct-observation instrument to assess trainee performance of a medication management session.


The Psychopharmacotherapy-Structured Clinical Observation (P-SCO) instrument was developed based on multiple sources of expertise and then implemented in 4 university-based outpatient medication management clinics with 7 faculty supervising 17 third-year residents. After each observation by a faculty member of a medication management session, residents received feedback in writing (the completed P-SCO) and verbally in person. Targets were 8 P-SCO observations per academic year per resident (or 0.67 per month) and 16 observations per year completed by each faculty (or 1.3 per month). Qualitative thematic analysis was employed to compare the frequency, specificity, type (reinforcing vs. corrective), and content of comments documented on the P-SCO forms to midpoint and end of rotation global assessments by the same faculty for the same residents in the same rotation.


Faculty completed 2.4 (SD=1,2) P-SCOs per month during the study period. Each resident received 1.1 (SD=0.53) P-SCO observations per month. Faculty and residents completed significantly more observations than targeted (p=0.03 and p=0.003, respectively). Two percent of the P-SCOs had no written comments. Less than 3% of the P-SCO comments were nonspecific compared with 43% for the global assessments. Residents received, on average, 3.3 times more total, 2.6 times more reinforcing, and 5.3 times more corrective patient care specific comments on the P-SCO than on the global assessment (p<0.001). For the numerical ratings, residents received an average of 4.2 “exceeds expectations” and 1.7 “below expectations” ratings on P-SCOs compared with 2.6 and 0, respectively, on global assessments (p<0.02).


Faculty can feasibly use the P-SCO instrument in a training clinic. Compared with traditional global assessment, the P-SCO provided much more specific feedback information, a better balance of corrective to re-enforcing comments, and a greater spread of ratings related to competency in pharmacodverbally