Skip to main content
SpringerLink
Log in

Visual Inspection of Data Revisited: Do the Eyes Still Have It?

  • On Methods
  • Published:
The Behavior Analyst Aims and scope Submit manuscript

Abstract

In behavior analysis, visual inspection of graphic information is the standard by which data are evaluated. Efforts to supplement visual inspection using inferential statistical procedures to assess intervention effects (e.g., analysis of variance or time-series analysis) have met with opposition. However, when serial dependence is present in the data, the use of visual inspection by itself may prove to be problematic. Previously published reports demonstrate that autocorrelated data influence trained observers’ ability to identify level treatment effects and trends that occur in the intervention phase of experiments. In this report, four recent studies are presented in which autoregressive equations were used to produce point-to-point functions to simulate experimental data. In each study, various parameters were manipulated to assess trained observers’ responses to changes in point-to-point functions from the baseline condition to intervention. Level shifts over baseline behavior (treatment effect), as well as no change from baseline (no treatment effect or trend), were most readily identified by observers, but trends were rarely recognized. Furthermore, other factors previously thought to augment and improve observers’ responses had no impact. Results are discussed in terms of the use of visual inspection and the training of behavior analysts.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Austin, J., & Mawhinney, T. (1996, May). Relations among data analysts’ discriminations between baseline and intervention streams, methods of data presentation, and analyses. Paper presented at the 22nd annual convention of the Association for Behavior Analysis, San Francisco.

  • Baer, D. M. (1977). Perhaps it would be better not to know everything. Journal of Applied Behavior Analysis, 10, 167–172.

    Article  PubMed  PubMed Central  Google Scholar 

  • Bailey, D. B. (1984). Effects of lines of progress and semilogarithmic charts on ratings of charted data. Journal of Applied Behavior Analysis, 17, 359–365.

    Article  PubMed  PubMed Central  Google Scholar 

  • Bakan, D. (1954). A generalization of Sidman’s results on group and individual functions, and a criterion. Psychological Bulletin, 51, 63–64.

    Article  PubMed  Google Scholar 

  • Busk, P. L., & Marascuilo, L. A. (1988). Autocorrelation in single-subject research: A counterargument to the myth of no autocorrelation. Behavioral Assessment, 10, 229–242.

    Google Scholar 

  • Cleveland, W. S., & McGill, R. (1984). Graphical perception: Theory, experimentation, and application to the development of graphical methods. Journal of the American Statistical Association, 79, 531–554.

    Article  Google Scholar 

  • Cleveland, W. S., & McGill, R. (1985). Graphical perception and graphical methods for analyzing scientific data. Science, 229, 828–833.

    Article  PubMed  Google Scholar 

  • Cleveland, W. S., & McGill, R. (1987). Graphical perception: The visual decoding of quantitative information on graphical displays of data. Journal of the Royal Statistical Society, A, 150, 192–229.

    Article  Google Scholar 

  • Cohen, J. (1988). Statistical power analysis for the behavioral sciences. Hillsdale, NJ: Erlbaum.

    Google Scholar 

  • Crosbie, J. (1987). The inability of the binomial test to control Type I erro with single-subject data. Behavioral Assessment, 9, 141–150.

    Google Scholar 

  • Crosbie, J. (1989). The inappropriateness of the C statistic for assessing stability or treatment effects with single-subject data. Behavioral Assessment, 11, 315–325.

    Google Scholar 

  • Crosbie, J. (1993). Interrupted time-series analysis with brief single-subject data. Journal of Consulting and Clinical Psychology, 61, 966–974.

    Article  PubMed  Google Scholar 

  • Deming, W. E. (1953). On the distinction between enumerative and analytic surveys. Journal of the American Statistics Association, 48, 244–255.

    Article  Google Scholar 

  • DeProspero, A., & Cohen, S. (1979). Inconsistent visual analyses of intrasubject data. Journal of Applied Behavior Analysis, 12, 573–579.

    Article  PubMed  PubMed Central  Google Scholar 

  • Ericsson, A. K., & Simon, H. A. (1980). Verbal reports as data. Psychological Review, 87, 215–281.

    Article  Google Scholar 

  • Estes, W. K. (1956). The problem of inference from curves based on group data. Psychological Bulletin, 53, 134–140.

    Article  PubMed  Google Scholar 

  • Fisch, G. S., & Porto, A. F. (1994). Visual inspection of data: Does the eyeball fit the trend? In B. E. Rogowitz & J. P. Allebach (Eds.), Human vision, visual processing, and digital display V (pp. 268–276). Bellingham, WA: SPIE Volume 2179.

    Chapter  Google Scholar 

  • Fisch, G. S., & Schneider, R. (1993). Visual inspection of data: Placement of point-to-point graphs affects discrimination of trends and treatment effects. American Statistical Association Proceedings: Section on Statistical Graphics, 55–58.

  • Furlong, M. J., & Wampold, B. E. (1982). Intervention effects and relative variation as dimensions in experts’ use of visual inference. Journal of Applied Behavior Analysis, 15, 415–421.

    Article  PubMed  PubMed Central  Google Scholar 

  • Gentile, J. R., Roden, A. H., & Klein, R. D. (1972). An analysis-of-variance model for the intrasubject replication design. Journal of Applied Behavior Analysis, 5, 193–198.

    Article  PubMed  PubMed Central  Google Scholar 

  • Glass, G. V., Willson, V. L., & Gottman, J. M. (1975). Design and analysis of time series experiments. Boulder, CO: Colorado Associated University Press.

    Google Scholar 

  • Greenspan, P., & Fisch, G. S. (1992). Visual inspection of data: A statistical analysis of behavior. American Statistical Association Proceedings: Section on Statistical Graphics, 79–82.

  • Greenwood, K. M., & Matyas, T. A. (1990). Problems with the application of interrupted time series analysis for brief single-subject data. Behavioral Assessment, 12, 355–370.

    Google Scholar 

  • Grote, I., & Baer, D. M. (1996, May). Visual analysis of graphic data by scientists-in-training. Paper presented at the 22nd annual convention of the Association for Behavior Analysis, San Francisco.

  • Haccou, P., & Meelis, E. (1992). Statistical analysis of behavioural data: An approach based on time-structured models. Oxford, England: Oxford University Press.

    Google Scholar 

  • Hahn, G. J., & Meeker, W. Q. (1991). Statistical intervals: A guide for practitioners. New York: Wiley.

    Book  Google Scholar 

  • Harrop, J. W., & Velicer, W E (1985). A comparison of alternative approaches to the analysis of interrupted times-series. Multivariate Behavioral Research, 20, 27–44.

    Article  PubMed  Google Scholar 

  • Hartmann, D. P. (1977). Considerations in the choice of interobserver reliability estimates. Journal of Applied Behavior Analysis, 10, 103–116.

    Article  PubMed  PubMed Central  Google Scholar 

  • Hartmann, D. P., Gottmann, J. M., Jones, R. R., Gardner, W., Kazdin, A. E., & Vaught, R. S. (1980). Interrupted time-series analysis and its application to behavioral data. Journal of Applied Behavior Analysis, 13, 543–559.

    Article  PubMed  PubMed Central  Google Scholar 

  • Huitema, B. E. (1985). Autocorrelation in applied behavior analysis: A myth. Behavioral Assessment, 109–120.

  • Huitema, B. E. (1986). Autocorrelation in behavioral research: Wherefore art thou? In A. Poling & R. W. Fuqua (Eds.), Research methods in applied behavior analysis: Issues and advances (pp. 187–208). New York: Plenum Press.

    Chapter  Google Scholar 

  • Jones, R. R., Vaught, R. S., & Weinrott, M. (1977). Time-series analysis in operant research. Journal of Applied Behavior Analysis, 10, 151–166.

    Article  PubMed  PubMed Central  Google Scholar 

  • Jones, R. R., Weinrott, M., & Vaught, R. S. (1978). Effects of serial dependency on the agreement between visual and statistical inference. Journal of Applied Behavior Analysis, 11, 277–283.

    Article  PubMed  PubMed Central  Google Scholar 

  • Karp, H. J., & Fisch, G. S. (1996, May). Signal detection analysis of responses to graphic data produced by auto-correlation and least squares regression equations. Paper presented at the 22nd annual convention of the Association for Behavior Analysis, San Francisco.

  • Kazdin, A. E. (1977). Artifact, bias, and complexity of assessment: The ABCs of reliability. Journal of Applied Behavior Analysis, 10, 141–150.

    Article  PubMed  PubMed Central  Google Scholar 

  • Kazdin, A. E. (1978). Statistical analysis for single-case experimental designs. In M. Hersen & D. H. Barlow (Eds.), Single case experimental designs: Strategies for studying behavioral change (pp. 265–316). New York: Pergamon Press.

    Google Scholar 

  • Kelly, M. B. (1977). A review of the observational data-collection and reliability procedures reported in the Journal of Applied Behavior Analysis. Journal of Applied Behavior Analysis, 10, 97–101.

    Article  PubMed  PubMed Central  Google Scholar 

  • Keselman, H. J., & Leventhal, L. (1974). Concerning the statistical procedures enumerated by Gentile et al.: Another perspective. Journal of Applied Behavior Analysis, 7, 643–645.

    Article  PubMed  PubMed Central  Google Scholar 

  • Kratochwill, T., Alden, K., Demuth, D., Dawson, D., Panicucci, C., Arntson, P., McMurray, N., Hempstead, J., & Levin, J. (1974). A further consideration in the application of an analysis-of-variance model for the intrasubject replication design. Journal of Applied Behavior Analysis, 7, 629–633.

    Article  PubMed  PubMed Central  Google Scholar 

  • Kratochwill, T. R., & Wetzel, R. J. (1977). Observer agreement, credibility, and judgment: Some considerations in presenting observer agreement data. Journal of Applied Behavior Analysis, 10, 133–139.

    Article  PubMed  PubMed Central  Google Scholar 

  • Lewandowsky, S., & Spence, I. (1989). Discriminating strata in scatterplots. Journal of the American Statistical Association, 84, 682–688.

    Article  Google Scholar 

  • Magnusson, D., Bergman, L. R., Rudinger, G., & Törestad, B. (1991). Problems and methods in longitudinal research: Stability and change. Cambridge, England: Cambridge University Press.

    Book  Google Scholar 

  • Matyas, T. A., & Greenwood, K. M. (1990). Visual analysis of single-case time series: Effects of variability, serial dependence, and magnitude of intervention effects. Journal of Applied Behavior Analysis, 23, 341–351.

    Article  PubMed  PubMed Central  Google Scholar 

  • Matyas, T. A., & Greenwood, K. M. (1991). Problems in the estimation of autocorrelation in brief time series and some implications for behavioral data. Behavioral Assessment, 13, 137–157.

    Google Scholar 

  • McEwan, J. S. A. (1994). Optimising line graph aspect ratio. Unpublished doctoral dissertation, The University of Waikato, New Zealand.

    Google Scholar 

  • Michael, J. (1974). Statistical inference for individual organism research: Mixed blessing or curse? Journal of Applied Behavior Analysis, 7, 647–653.

    Article  PubMed  PubMed Central  Google Scholar 

  • Ottenbacher, K. J. (1986). Reliability and accuracy of visually analyzing graphed data from single-subject designs. American Journal of Occupational Therapy, 40, 464–469.

    Article  PubMed  Google Scholar 

  • Parsonson, B. S., & Baer, D. M. (1992). The visual analysis of data, and current research into the stimuli controlling it. In T. R. Kratochwill & J. R. Levin (Eds.), Single-case research design and analysis (pp. 15–41). Hillsdale, NJ: Erlbaum.

    Google Scholar 

  • Pfadt, A., Cohen, I. L., Sudhalter, V., Romanczyk, R. G., & Wheeler, D. J. (1992). Applying statistical process control to clinical data: An illustration. Journal of Applied Behavior Analysis, 25, 551–560.

    Article  PubMed  PubMed Central  Google Scholar 

  • Sharpley, C. F., & Alavosius, M. P. (1988). Autocorrelation in behavioral data: An alternative perspective. Behavioral Assessment, 10, 243–251.

    Google Scholar 

  • Shewhart, W. A. (1931). Economic control of quality of manufactured product. Princeton, NJ: Reinhold Co.

    Google Scholar 

  • Shrout, P. E. (1997). Should significance testing be banned? Introduction to a special section exploring the pros and cons. Psychological Science, 8, 1–2.

    Article  Google Scholar 

  • Sidman, M. (1952). A note on functional relations obtained from group data. Psychological Bulletin, 49, 263–269.

    Article  PubMed  Google Scholar 

  • Sidman, M. (1960). Tactics of scientific research: Evaluation of experimental data in psychology. New York: Basic Books.

    Google Scholar 

  • Skinner, B. F. (1938). The behavior of organisms. New York: Appleton-Century-Crofts.

    Google Scholar 

  • Spence, I. (1990). Visual psychophysics of simple graphical elements. Journal of Experimental Psychology: Human Perception and Performance, 16, 683–692.

    PubMed  Google Scholar 

  • Spence, I., & Lewandowsky, S. (1990). Graphical perception. In J. Fox & J. S. Long (Eds.), Modern methods of data analysis (pp. 13–57). Newbury Park, CA: Sage.

    Google Scholar 

  • Stigler, S. M. (1986). Estimating serial correlation by visual inspection of diagnostic plots. The American Statistician, 40, 111–116.

    Google Scholar 

  • Thoresen, C. E., & Elashoff, J. D. (1974). An analysis-of-variance model for intrasubject replication design: Some additional comments. Journal of Applied Behavior Analysis, 7, 639–641.

    Article  PubMed  PubMed Central  Google Scholar 

  • Toothaker, L. E., Banz, M., Noble, C., Camp, J., & Davis, D. (1983). N = 1 designs: The failure of ANOVA-based tests. Journal of Educational Statistics, 8, 289–309.

    Article  Google Scholar 

  • Velicer, W. F., & Harrop, J. (1983). The reliability and accuracy of time series model identification. Evaluation Review, 7, 551–560.

    Article  Google Scholar 

  • Wampold, B., & Furlong, M. (1981). The heuristics of visual inference. Behavioral Assessment, 3, 79–92.

    Google Scholar 

  • Wetherill, G. B., & Brown, D. W. (1991). Statistical process control: Theory and practice. London: Chapman and Hall.

    Book  Google Scholar 

  • Yeltman, A. R., Wildman, B. G., & Erickson, M. T. (1977). A probability-based formula for calculating interobserver agreement. Journal of Applied Behavior Analysis, 127–131.

Download references

Author information

Authors and Affiliations

  1. General Clinical Research Center, Yale University, 530 Hunter Building, 333 Cedar St., New Haven, Connecticut, 06520, USA

    Gene S. Fisch

Authors
  1. Gene S. Fisch
    View author publications

    You can also search for this author in PubMed Google Scholar

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Fisch, G.S. Visual Inspection of Data Revisited: Do the Eyes Still Have It?. BEHAV ANALYST 21, 111–123 (1998). https://doi.org/10.1007/BF03392786

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/BF03392786

Key words

Search

Navigation