Summary
The development of the Behavioral Coding System (BCS) used by the Social Learning Project has encompassed approximately 8 years of clinical and research experience with naturalistic observation as a clinical assessment tool. The BCS, while originally designed to accomplish certain broad purposes, illustrates a solution to an assessment task that should be applicable to other research and clinical settings in which naturalistic observation of family interactions are needed.
A variety of reliability analyses, ranging from traditional interobserver agreement among coders to generalizability analyses, have supported the measurement precision of the BCS scores, for their intended purposes. In conducting this series of investigations, certain problems in psychometric analysis of observation data have arisen and been documented. Most notably, the tradition of estimating reliability via interobserver agreement has been questioned, mainly on the grounds that behavioral complexity intrudes into such analyses in ways that suggest that current observer reliability estimates may be substantially biased. The usefulness of generalizability theory is argued, particularly for observational data collected under varying assessment conditions which may influence behavioral scores.
Three types of validity have been reported for BCS scores: content, concurrent, and construct validity. The BCS has favorably withstood these psychometric investigations, showing that the behavioral measures are justified on content grounds, that outside reports of behavior coincide satisfactorily with the BCS scores, and that expected behavioral changes following treatment are readily indexed by the BCS scores.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Antonovsky, H. F. A contribution to research in the area of mother-child relationship.Child Development, 1959,30, 37–51.
Azrin, N., Holtz, W., Ulrich, R., & Goldiamond, I. The control of conversation through reinforcement.Journal of Experimental Analysis Behavior, 1961,4, 25–30.
Becker, W. C. The relationship of factors in parental ratings of self and each other to the behavior of kindergarten children as rated by mothers, fathers, and teachers.Journal of Consulting Psychology, 1960,24, 507–527.
Bing, E. Effect of child-rearing practices on development of differential cognitive abilities.Child Development, 1963,34, 631–648.
Clement, P. W., & Milne, D. C. Group play therapy and tangible reinforcers used to modify the behavior of eight-year-old boys.Behaviour Research and Therapy, 1967,5, 301–312.
Collins, R. C.The treatment of disruptive classroom behavior problems by employment of a partial-milieu consistency program. Unpublished doctoral dissertation, University of Oregon, 1966.
Crandall, V. J., & Preston, A. Verbally expressed needs and over maternal behaviors.Child Development, 1961,32, 261–270.
Devine, V. T.The coercion process: A laboratory analogue. Unpublished doctoral dissertation, State University of New York at Stony Brook, 1971.
Harris, A. M.Observer effects on family interaction. Unpublished doctoral dissertation, University of Oregon, 1969.
Hendriks, A. F. C. J.Reported versus observed deviancy. Unpublished manuscript, University of Nijmegen, Netherlands, 1972.
Honig, A. S., Tannenbaum, J., & Caldwell, B. M.Maternal behavior in verbal report and in laboratory observation. Paper presented at the meeting of the American Psychological Association, San Francisco, September 1968.
Johnson, S. M., & Bolstad, O. D. Methodological issues in naturalistic observation: Some problems and solutions for field research. In L. A. Hamerlynck, L. C. Handy, & E. J. Mash (Eds.),Behavior change: Methodology concepts and practice. Champaign, Illinois: Research Press, 1973. Pp. 7–68.
Jones, R. R., Reid, J. B., & Patterson, G. R. Naturalistic observations in clinical assessment. In P. McReynolds (Ed.),Advances in psychological assessment (VoL 3). San Francisco: Jossey-Bass, 1975. Pp. 42–95.
Kent, R. N., O'Leary, K. D., Diament, C., & Dietz, A. Expectation biases in observational evaluation of therapeutic change.Journal of Consulting and Clinical Psychology, 1974,42, 774–780
Lobitz, W. C.Parental response sets and the behavior of deviant and nondeviant children during naturalistic observation. Unpublished doctoral dissertation, University of Oregon, 1973.
Lobitz, G., & Johnson, W. M.Normal versus deviant: Fact or fantasy. Paper presented at the meeting of the Western Psychological 113-1iation, Portland, Oregon, April 1972.
Patterson, G. R. An application of conditioning techniques to the contiol of a hyperactive child. In L. P. Ullmann & L. Krasner (Eds.),Case studies in behavior modification. New York: Holt, Rinehar' & Winston, 1965. Pp. 370–375.
Patterson, G. R. Interventions for boys with conduct problems: Multiple settings, treatments, and criteria.Journal of Consulting and Clinical Psychology, 1974,42, 471–481.
Patterson, G. R. Retraining of aggressive boys by their parents: Review of recent literature and follow-up evaluation. In F. Lowy (Ed.), Symposium on the Seriously Disturbed Preschool Child,Canadian Psychiatric Association Journal, 1974,19, 142–161.
Patterson, G. R. Stimulus control in natural settings. In J. DeWit & W. Hartup (Eds.),Determinants and origins of aggressive behaviors. The Hague, Netherlands: Mouton Press, 1974. Pp. 391–400.
Patterson, G. R. Multiple evaluations of a parent training program. In T. Thompson & W. S. Dockens III (Eds.),Applications of behavior modification. New York: Academic Press, 1975. Pp. 299–322.
Patterson, G. R., & Cobb, J. A. A dyadic analysis of “aggressive” behaviors. In J. P. Hill (Ed.),Minnesota symposia on child psychology (Vol. 5). Minneapolis: University of Minnesota Press, 1971. Pp. 72–129.
Patterson, G. R., & Cobb, J. A. Stimulus control for classes of noxious behaviors. In J. F. Knutson (Ed.),The control of aggression: Implications from basic research. Chicago: Aldine, 1973. Pp. 144–199. (See NAPS Document #02107 for 13 pages of supplementary material. Order from AS1S/NAPS, c/o Microfiche Publications, 440 Park Avenue South, New York, N.Y. 10016. Remit in advance $5.45 for photocopies or $1.50 for microfiche. Make checks payable to Microfiche Publications.)
Patterson, G. R., Cobb, J. A., & Ray, R. S. A social engineering technology for retraining the families of aggressive boys. In H. Adams & I. P. Unikel (Eds.),Issues and trends in behavior therapy. Springfield, Illinois: Charles. C Thomas, 1973. Pp. 139–224.
Patterson, G. R., & Fagot, B. I. Selective responsiveness to social reinforcers and deviant behavior in children.Psychological Record, 1967,17, 369–378.
Patterson, G. R., McNeal, S., Hawkins, N., & Phelps, R. Reprogramming the social environment.Journal of Child Psychology and Psychiatry, 1967,8, 181–195.
Patterson, G. R., Ray, R. S., Shaw, D. A., & Cobb, J. A.A manual for coding of family interactions. 1969 (rev.). (See Document #01234 for 33 pages of supplementary materials. Order from ASIS/NAPS, c/o Microfiche Publications, 440 Park Avenue South. New York. N.Y. 10016. Remit in advance $5.45 for photocopies or $1.50 for microfiche. Make checks payable to Microfiche Publications.)
Patterson, G. R., & Reid, J. B. Intervention for families of aggressive boys: A replication study.Behaviour Research and Therapy, 1973,11, 383–394.
Radkc-Yarrow, M. Problems of methods in parent child research.Child Development, 1963,34, 215–226.
Reid, J. B.Reciprocity in family interaction. Unpublished doctoral dissertation, University of Oregon, 1967.
Reid, J. B. (Ed.).A social learning approach to family interaction (Vol. 2).A manual for coding family interactions. Eugene, Oregon: Castalia Publishing Company, 1977.
Reid, J. B., & Hendriks, A. F. C. J. A preliminary analysis of the effectiveness of direct home intervention for treatment of pre-delinquent boys who steal. In L. A. Hamerlynck, L. C. Handy, & E. J. Mash (Eds.),Behavior change: Methodology concepts and practice. Campaign, Illinois: Research Press, 1973. Pp. 209–219.
Rosenthal, R.Experimenter effects in behavioral research. New York: Appleton-CenturyCrofts, 1966.
Sears, R. R. Comparison of interviews with questionnaires for measuring mothers' attitudes toward sex and aggression.Journal of Personality and Social Psychology, 1965,2, 37–44.
Skindrud, K. D.An evaluation of observer bias in experimental-field studies of social interaction. Unpublished doctoral dissertation, University of Oregon, 1972.
Skindrud, K. D. Field evaluation of observer bias under overt and covert monitoring. In L. Hamerlynck, L. Handy, & E. Mash (Eds.),Behavior change: Methodology concepts and practice. Champaign, Illinois: Research Press, 1973.
Smith, H. T. A comparison of interview and observation measures of mother behavior.Journal of Abnormal and Social Psychology, 1958,57, 278–282.
Taplin, P. S., & Reid, J. B. Effects of instructional set and experimenter influence on observer reliability.Child Development, 1973,44, 547–554.
Walther, H. I., & Gilmore, S. K. Placebo versus social learning effect in parent training procedures designed to alter the behavior of aggressive boys.Behavior Therapy, 1973,4, 361–377.
White, G.The effects of observer presence on mother and child behavior. Unpublished doctoral dissertation, University of Oregon, 1972.
Author information
Authors and Affiliations
Additional information
Excerpts and abstracts from chapters by Jones, Reid, and Patterson (1975, pp. 42–95), and from Reid (1977).
The assessment procedures were developed as part of an extended series of grants from the Section on Crime and Delinquency, National Institute of Mental Health.
J. B. Reid (1977) has recently edited a manual which presents a much fuller report of the topics covered in this report, including: a more extensive literature review, operational definitions of code categories, normative data, video training tapes, and procedures for training observers.
Rights and permissions
About this article
Cite this article
Patterson, G.R. Naturalistic observation in clinical assessment. J Abnorm Child Psychol 5, 309–322 (1977). https://doi.org/10.1007/BF00913701
Issue Date:
DOI: https://doi.org/10.1007/BF00913701