Skip to main content
SpringerLink
Log in

The Reliability of Psychiatric Diagnosis

  • Chapter
Issues in Diagnostic Research

Abstract

In science, one needs to be able to understand and repeat another’s work. In psychopathology this is difficult, since our descriptions of patients are often ambiguous and idiosyncratic (Elkin, 1947). Measurement research in psychopathology corresponds to the development of laboratory tests in clinical medicine; it is often tedious but indispensable. This chapter first surveys some methodological and statistical problems that arise in diagnostic reliability research. Recent developments in the statistics of diagnostic reliability are reviewed. The outcomes of studies of diagnostic reliability are then surveyed. Ongoing studies related to diagnosis are touched on. The reader is cautioned against overoptimism about the attainable certainty of psychiatric diagnosis.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
eBook
USD 16.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  • American Psychiatric Association. (1968). Diagnostic and Statistical Manual of Mental Disorders 2nd edition. Washington, DC: Author.

    Google Scholar 

  • American Psychiatric Association. (1980). Diagnostic and Statistical Manual of Mental Disorders 3rd edition. Washington, DC: Author.

    Google Scholar 

  • Andreasen, N.C., McDonald-Scott, P., Grove, W.M., Keller, M.D., Shapiro, R.W., & Hirschfeld, R.M.A. (1982). Assessment of reliability in multi-center collaborative research using a videotape approach. American Journal of Psychiatry139 876–882.

    PubMed  Google Scholar 

  • Andreasen, N.C., Grove, W.M., Shapiro, R.W., Keller, M.B., Hirschfeld, R.M.A., & McDonald-Scott, P. (1981)

    Google Scholar 

  • Reliability of lifetime diagnosis: A multicenter collaborative perspective. Archives of General Psychiatry 38 400–405.

    Google Scholar 

  • Beck, AT., Ward, C.H., Mendelson, M., Mock, J.E., & Erbaugh, J.K. (1962). Reliability of psychiatric diagnoses: 2. A study of consistency of clinical judgements and ratings. American Journal of Psychiatry119 351–357.

    PubMed  Google Scholar 

  • Burman, M.A, Karno, M., Hough, R.L., Escobar, J.I., & Forsythe, A.B. (1983). The Spanish Diagnostic Interview Schedule: Reliability and comparison with clinical diagnoses. Archives of General Psychiatry40 1189–1195.

    Article  Google Scholar 

  • Carey, G., & Gottesman, I.I. (1978). Reliability and validity in binary ratings: Areas of common misunderstanding in diagnosis and symptom ratings. Archives of General Psychiatry35 1454–1459.

    Article  PubMed  Google Scholar 

  • Cloninger, C.R., Miller, J.P., Wette, R., Martin, R.I., & Guze, S.B. (1979). The evaluation of diagnostic concordance in follow-up studies: 1. A general model of causal analysis and a methodologicalcritique. Journal of Psychiatric Research15 85–106.

    Article  PubMed  Google Scholar 

  • Cohen, J. (1960). A coefficient of agreement for nominal scales. Educational and Psychological Measurement 2037–46.

    Article  Google Scholar 

  • Coryell, W., Cloninger, C.R., & Reich, T. (1978) Clinical assessment: Use of nonphysician interviewers. Journal of Nervous and Mental Disease166599–606.

    Article  PubMed  Google Scholar 

  • DiNardo, P.A., O’Brien, G.T., Barlow, D.H., Waddell, M.T., & Blanchard, E.B. (1983). Reliability ofDSM-III anxiety disorder categories using a new structured interview. Archives of General Psychiatry40 1170–1078.

    Article  Google Scholar 

  • Elkin, F. (1947). Specialists interpret the case of Harold Holzer. Journal of Abnormal and Social Psychology44 272–276.

    Google Scholar 

  • Fleiss, J.L. (1981) Balanced incomplete block designs for interrater reliability studies. Applied Psychological Measurement5 105–112.

    Article  Google Scholar 

  • Fleiss, J.L., & Cohen, J. (1973). The equivalence of weighted kappa and the intraclass correlation coefficient as measures of reliability. Educational and Psychological Measurement, 33 613–619.

    Article  Google Scholar 

  • Fleiss, J.L., & Cuzick, J. (1979). The reliability of dichotomous judgments: Unequal numbers of judges per Applied Psychological Measurement, 3 537–542.

    Article  Google Scholar 

  • Foulds, G.A. (1955). The reliability of psychiatric, and the validity of psychological, diagnoses. Journal of Mental Science, 101 851–862.

    PubMed  Google Scholar 

  • Galen, G.S., & Gambino, S.R. (1975). Beyond normality: The predictive value and efficiency of medical diagnoses. New York: Wiley.

    Google Scholar 

  • Helzer, J.E., & Coryell, W. (1983). More on DSM-III: How consistent are precise criteria? (Ediior’ml) Biological Psychiatry, 18 1201–1203.

    Google Scholar 

  • Helzer, J.E., Clayton, P.J., Pambakian, R., Reich, T., Woodruff, R.A., Jr., & Reveley, M.A. (1977). Reliability of psychiatric diagnosis: II. The test/retest reliability of diagnostic classification. Archives of General Psychiatry, 34 136–141.

    Article  PubMed  Google Scholar 

  • Helzer, J.E., Robins, L.N., Croughan, J.L., & Welner, A. (1981). Renard Diagnostic Interview: Its reliability and procedural validity with physicians and lay interviewers. Archives of General Psychiatry, 38 393–398.

    Article  PubMed  Google Scholar 

  • Helzer, J.E., Robins, L.N., McEvoy, L.T., Spitznagel, E.L., Stoltzman, R.K., Farmer, A.L., & Brockington, I.F. (1985). Results of the St. Louis ECA physician reexamination study of the Diagnostic Interview Schedule. Archives of General Psychiatry, 42 657–666.

    Article  PubMed  Google Scholar 

  • Helzer, J.E., Stoltzman, R.K., Farmer, A., Brockington, I.F., Plesons, D., Singerman, B., & Works, J. (1985) Comparison with a DIS/DSM-III based physician réévaluation in St. Louis. In W.W. Eaton & L.G. Kessler (Eds.), Epidemiologic field methods in psychiatry: The NIMH Epidemiologic Catchment Area program. New York: Academic Press.

    Google Scholar 

  • Hesselbrock, V., Stabenau, J., Hesselbrock, M., Mirkin, P., & Meyer, R. (1982). A comparison of two interview schedules: The Schedule for Affective Disorders and Schizophrenia- Lifetime and the National Institute for Mental Health Diagnostic Interview Schedule. Archives of General Psychiatry, 39 674–677.

    Article  PubMed  Google Scholar 

  • Hostetter, AM., Egeland, J.A., & Endicott, J. (1983). Amish Study, II: Consensus diagnoses and reliability results. American Journal of Psychiatry, 140 62–71.

    PubMed  Google Scholar 

  • Hyler, S.E., Williams, J.B.W., & Spitzer, R.L. (1982). Reliability in the DSM-III field trials: Interview V. case summary. Archives of General Psychiatry, 39 1275–1278.

    Article  PubMed  Google Scholar 

  • Johnson, J.H., Klingler, D.E., Giannetti, R.A., & Williams, T.A (1980). The reliability of diagnoses by technician, computer, and algorithm. Journal of Clinical Psychology, 36 447–451.

    PubMed  Google Scholar 

  • Katz, M.M., Secunda, S.K, Hirschfeld, R.MA, Jr. & Koslow, S.R. (1979). NIMH Clinical Research Branch Collaborative Program on the Psychobiology of Depression. Archives of General Psychiatry, 36 765–771.

    Article  PubMed  Google Scholar 

  • Keller, M.B., Lavori, P.W., Andreasen, N.C., Grove, W.M., Shapiro, R.W., Sheftner, W., & McDonald-Scott, P. (1981). Test-retest reliability of assessing psychiatrically ill patients in a multi-center design. Journal of Psychiatric Research, 16 213–227.

    Article  PubMed  Google Scholar 

  • Keller, M.B., Lavori, P., McDonald-Scott, P., Scheftner, W.A, Andreasen, N.C., Shapiro, R.W., & Croughan, J. (1981). Reliability of lifetime diagnoses and symptoms in patients with a current psychiatric disorder. Journal of Psychiatric Research, 16 229–240.

    Article  PubMed  Google Scholar 

  • Kendell, R.E., Cooper, J.E., Gourlay, AJ., Copeland, J.R.M., Sharpe, L., & Gurland, B.J. (1971). Diagnostic criteria of American and British psychiatrists, Archives of General Psychiatry, 25 123–130.

    Article  PubMed  Google Scholar 

  • Kraemer, H.C. (1978). Ramifications of a population model for K as a coefficient of reliability. Psychometrika, 44 461–472.

    Article  Google Scholar 

  • Kreitman, N., Sainsbury, P., Morrissey, J., Towers, J., & Scrivener, J. (1961). The reliability of psychiatric assessment: An analysis. Journal of Mental Science, 107 876–886.

    PubMed  Google Scholar 

  • Landis, J.R., & Koch, G.G. (1977). A one-way components of variance model for categorical data. Biometrics, 33 671–679.

    Article  Google Scholar 

  • Martin, R.L., Cloninger, C.R., & Guze, S.B. (1979). The evaluation of diagnostic concordance in follow-up studies: 11. A blind, prospective follow-up of female criminals. Journal of Psychiatric Research, 15 107–125.

    Article  PubMed  Google Scholar 

  • Masserman, J.H., & Carmichael, H.T. (1938). Diagnosis and prognosis in psychiatry: With a follow-up study of the results of short-term general hospital therapy of psychiatric cases. Journal of Mental Science, 84 893–946.

    Google Scholar 

  • Mazure, C., & Gershon, E.S. (1979). Blindness and reliability in lifetime psychiatric diagnosis. Archives of General Psychiatry, 36 521–525.

    Article  PubMed  Google Scholar 

  • Mellsop, G., Varghese, F., Joshua, S., & Hicks, A (1982). The reliability of Axis II of DSM-III: A report of a clinical study. American Journal of Psychiatry, 139 1360–1361.

    PubMed  Google Scholar 

  • Norris, v. (1959). Mental illness in London. London: Chapman & Hall.

    Google Scholar 

  • Regier, D.A., Myers, J.K., Kramer, M., Robins, L.N., Blazer, D.G., Hough, R.L., Eaton, W.W., & Locke, B.Z. (1984). The NIMH Epidemiologic Catchment Area program. Archives of General Psychiatry, 41 934–941.

    Article  PubMed  Google Scholar 

  • Robins, L.N., Helzer, J.E., Croughan, J., & Ratcliff, K.S. (1981). National Institute of Mental Health Diagnostic Interview Schedule: Its history, characteristics, and validity. Archives of General Psychiatry, 38 381–389.

    Article  PubMed  Google Scholar 

  • Robins, L.N., Helzer, J.E., Ratcliff, K.S., & Seyfried, W. (1982). Reliability and validity of the Diagnostic Interview Schedule, Version II: DSM-III diagnoses. Psychological Medicine, 12 855–870.

    Article  PubMed  Google Scholar 

  • Rosenzweig, N., Vandenberg, S.G., Moore, K., & Dukay, A (1961). A study of the reliability of the mental status examination,American Journal of Psychiatry, 117 1102–1108.

    Google Scholar 

  • Rounsaville, B.J., Cacciola, J., Weissman, M.M., & Kleber, H.D. (1982). Diagnostic concordance in a follow-up study of opiate addicts. Journal of Psychiatric Research, 16 191- 201.

    Google Scholar 

  • Rounsaville, B.J., Rosenberger, P., Wilber, C., Weissman, M.M., & Kleber, H.D. (1980). A comparison of the SADS/RDC and the DSM-III: Diagnosing drug abusers. Journal of Nervous and Mental Disease 765,90–97.

    Google Scholar 

  • Sandifer, M.G., Jr., Pettus, C., & Quade, D. (1964). A study of psychiatric diagnosis. Journal of Nervous and Mental Disease, 39 350–356.

    Google Scholar 

  • Schmidt, H.O., & Fonda, C.P. (1956). The reliability of psychiatric diagnosis: A new look. Journal of Abnormal and Social Psychology, 52 262–267.

    Article  Google Scholar 

  • Spitzer, R.L. (1983). Psychiatric diagnosis: Are clinicians still necessary? Comprehensive Psychiatry, 24 399–411.

    Article  PubMed  Google Scholar 

  • Spitzer, R.L., Endicott, J., Cohen, J., & Fleiss, J.L. (1974). Constraints on the validity of computer diagnosis. Archives of General Psychiatry, 31 197–203.

    Article  PubMed  Google Scholar 

  • Spitzer, R.L., Endicott, J., Robins, E., Kuriansky, J., & Gurland, B. (1975). Preliminary report of the reliability of Research Diagnostic Criteria applied to psychiatric case records. In A. Sudilovsky, S. Gershon, & B. Beer (Eds.), Predictability in psychopharmacology: Preclinical and clinical correlations (pp. 1–47). New York: Raven.

    Google Scholar 

  • Spitzer, R.L., Endicott, J., & Robins, E. (1978). Research Diagnostic Criteria: Rationale and Reliability. Archives of General Psychiatry, 35 773–782.

    Article  PubMed  Google Scholar 

  • Spitzer, R.L., Forman, J.B.W., & Nee, J. (1979). DSM-III field trials: I. Initial interrater diagnostic reliability. American Journal of Psychiatry, 136 815–817.

    Google Scholar 

  • Spitznagel, E.L., & Helzer, J.E. (1985). A proposed solution to the base rate problem in the kappa statistics. Archives of General Psychiatry, 42 725–728.

    Article  PubMed  Google Scholar 

  • Standage, KR. (1979). The use of Schneider’s typology for the diagnosis of personality disorders—An examination of reliability. British Journal of Psychiatry, 135 238–242.

    Article  PubMed  Google Scholar 

  • Stangl, D., Pfohl, B., Zimmerman, M., Bowers, W., & Corenthal, C. (1986). A structured interview for the DSM-III personality disorders: A preliminary study. Archives of General Psychiatry, 42 591–596.

    Article  Google Scholar 

  • Uebersax, J.S. (1983). A design-independent method for measuring the reliability of psychiatric diagnosis. Journal of Psychiatric Research, 4 335–342.

    Google Scholar 

  • Ward, C.H., Beck, AT., Mendelson, M., Mock, J.E., & Erbaugh, J.K (1962). The psychiatric nomenclature: Reasons for diagnostic disagreement. Archives of General Psychiatry, 7 198–205.

    Article  PubMed  Google Scholar 

  • Westermeyer, J., & Sines, L. (1979). Reliability of cross-cultural psychiatric diagnosis with an assessment of two rating contexts. Journal of Psychiatric Research, 15 199–213.

    Article  PubMed  Google Scholar 

  • Wing, J.K, Birley, J.L.T., Cooper, J.E., Graham, P., & Isaacs, AD. (1967). Reliability of a procedure for measuring and classifying present psychiatric stateBritish Journal of Psychiatry, 113 499–515.

    Article  PubMed  Google Scholar 

  • World Health Organization. (1973). International Pilot Study of Schizophrenia, Vol I Geneva: Author.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Deparment of Psychiatry, University of Iowa, 52242, Iowa City, IA, USA

    William M. Grove

Authors
  1. William M. Grove
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Western Psychiatric Institute and Clinic, University of Pittsburgh School of Medicine Pittsburgh, Pennsylvania, USA

    Cynthia G. Last  & Michel Hersen  & 

Rights and permissions

Reprints and permissions

Copyright information

© 1987 Plenum Press, New York

About this chapter

Cite this chapter

Grove, W.M. (1987). The Reliability of Psychiatric Diagnosis. In: Last, C.G., Hersen, M. (eds) Issues in Diagnostic Research. Springer, Boston, MA. https://doi.org/10.1007/978-1-4684-1265-9_3

Download citation

  • DOI: https://doi.org/10.1007/978-1-4684-1265-9_3

  • Publisher Name: Springer, Boston, MA

  • Print ISBN: 978-1-4684-1267-3

  • Online ISBN: 978-1-4684-1265-9

  • eBook Packages: Springer Book Archive

Publish with us

Policies and ethics

Search

Navigation