The Behavior Analyst

, Volume 21, Issue 1, pp 111–123 | Cite as

Visual Inspection of Data Revisited: Do the Eyes Still Have It?

  • Gene S. Fisch
On Methods

Abstract

In behavior analysis, visual inspection of graphic information is the standard by which data are evaluated. Efforts to supplement visual inspection using inferential statistical procedures to assess intervention effects (e.g., analysis of variance or time-series analysis) have met with opposition. However, when serial dependence is present in the data, the use of visual inspection by itself may prove to be problematic. Previously published reports demonstrate that autocorrelated data influence trained observers’ ability to identify level treatment effects and trends that occur in the intervention phase of experiments. In this report, four recent studies are presented in which autoregressive equations were used to produce point-to-point functions to simulate experimental data. In each study, various parameters were manipulated to assess trained observers’ responses to changes in point-to-point functions from the baseline condition to intervention. Level shifts over baseline behavior (treatment effect), as well as no change from baseline (no treatment effect or trend), were most readily identified by observers, but trends were rarely recognized. Furthermore, other factors previously thought to augment and improve observers’ responses had no impact. Results are discussed in terms of the use of visual inspection and the training of behavior analysts.

Key words

visual inspection data analysis statistical analysis 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Austin, J., & Mawhinney, T. (1996, May). Relations among data analysts’ discriminations between baseline and intervention streams, methods of data presentation, and analyses. Paper presented at the 22nd annual convention of the Association for Behavior Analysis, San Francisco.Google Scholar
  2. Baer, D. M. (1977). Perhaps it would be better not to know everything. Journal of Applied Behavior Analysis, 10, 167–172.CrossRefPubMedPubMedCentralGoogle Scholar
  3. Bailey, D. B. (1984). Effects of lines of progress and semilogarithmic charts on ratings of charted data. Journal of Applied Behavior Analysis, 17, 359–365.CrossRefPubMedPubMedCentralGoogle Scholar
  4. Bakan, D. (1954). A generalization of Sidman’s results on group and individual functions, and a criterion. Psychological Bulletin, 51, 63–64.CrossRefPubMedGoogle Scholar
  5. Busk, P. L., & Marascuilo, L. A. (1988). Autocorrelation in single-subject research: A counterargument to the myth of no autocorrelation. Behavioral Assessment, 10, 229–242.Google Scholar
  6. Cleveland, W. S., & McGill, R. (1984). Graphical perception: Theory, experimentation, and application to the development of graphical methods. Journal of the American Statistical Association, 79, 531–554.CrossRefGoogle Scholar
  7. Cleveland, W. S., & McGill, R. (1985). Graphical perception and graphical methods for analyzing scientific data. Science, 229, 828–833.CrossRefPubMedGoogle Scholar
  8. Cleveland, W. S., & McGill, R. (1987). Graphical perception: The visual decoding of quantitative information on graphical displays of data. Journal of the Royal Statistical Society, A, 150, 192–229.CrossRefGoogle Scholar
  9. Cohen, J. (1988). Statistical power analysis for the behavioral sciences. Hillsdale, NJ: Erlbaum.Google Scholar
  10. Crosbie, J. (1987). The inability of the binomial test to control Type I erro with single-subject data. Behavioral Assessment, 9, 141–150.Google Scholar
  11. Crosbie, J. (1989). The inappropriateness of the C statistic for assessing stability or treatment effects with single-subject data. Behavioral Assessment, 11, 315–325.Google Scholar
  12. Crosbie, J. (1993). Interrupted time-series analysis with brief single-subject data. Journal of Consulting and Clinical Psychology, 61, 966–974.CrossRefPubMedGoogle Scholar
  13. Deming, W. E. (1953). On the distinction between enumerative and analytic surveys. Journal of the American Statistics Association, 48, 244–255.CrossRefGoogle Scholar
  14. DeProspero, A., & Cohen, S. (1979). Inconsistent visual analyses of intrasubject data. Journal of Applied Behavior Analysis, 12, 573–579.CrossRefPubMedPubMedCentralGoogle Scholar
  15. Ericsson, A. K., & Simon, H. A. (1980). Verbal reports as data. Psychological Review, 87, 215–281.CrossRefGoogle Scholar
  16. Estes, W. K. (1956). The problem of inference from curves based on group data. Psychological Bulletin, 53, 134–140.CrossRefPubMedGoogle Scholar
  17. Fisch, G. S., & Porto, A. F. (1994). Visual inspection of data: Does the eyeball fit the trend? In B. E. Rogowitz & J. P. Allebach (Eds.), Human vision, visual processing, and digital display V (pp. 268–276). Bellingham, WA: SPIE Volume 2179.CrossRefGoogle Scholar
  18. Fisch, G. S., & Schneider, R. (1993). Visual inspection of data: Placement of point-to-point graphs affects discrimination of trends and treatment effects. American Statistical Association Proceedings: Section on Statistical Graphics, 55–58.Google Scholar
  19. Furlong, M. J., & Wampold, B. E. (1982). Intervention effects and relative variation as dimensions in experts’ use of visual inference. Journal of Applied Behavior Analysis, 15, 415–421.CrossRefPubMedPubMedCentralGoogle Scholar
  20. Gentile, J. R., Roden, A. H., & Klein, R. D. (1972). An analysis-of-variance model for the intrasubject replication design. Journal of Applied Behavior Analysis, 5, 193–198.CrossRefPubMedPubMedCentralGoogle Scholar
  21. Glass, G. V., Willson, V. L., & Gottman, J. M. (1975). Design and analysis of time series experiments. Boulder, CO: Colorado Associated University Press.Google Scholar
  22. Greenspan, P., & Fisch, G. S. (1992). Visual inspection of data: A statistical analysis of behavior. American Statistical Association Proceedings: Section on Statistical Graphics, 79–82.Google Scholar
  23. Greenwood, K. M., & Matyas, T. A. (1990). Problems with the application of interrupted time series analysis for brief single-subject data. Behavioral Assessment, 12, 355–370.Google Scholar
  24. Grote, I., & Baer, D. M. (1996, May). Visual analysis of graphic data by scientists-in-training. Paper presented at the 22nd annual convention of the Association for Behavior Analysis, San Francisco.Google Scholar
  25. Haccou, P., & Meelis, E. (1992). Statistical analysis of behavioural data: An approach based on time-structured models. Oxford, England: Oxford University Press.Google Scholar
  26. Hahn, G. J., & Meeker, W. Q. (1991). Statistical intervals: A guide for practitioners. New York: Wiley.CrossRefGoogle Scholar
  27. Harrop, J. W., & Velicer, W E (1985). A comparison of alternative approaches to the analysis of interrupted times-series. Multivariate Behavioral Research, 20, 27–44.CrossRefPubMedGoogle Scholar
  28. Hartmann, D. P. (1977). Considerations in the choice of interobserver reliability estimates. Journal of Applied Behavior Analysis, 10, 103–116.CrossRefPubMedPubMedCentralGoogle Scholar
  29. Hartmann, D. P., Gottmann, J. M., Jones, R. R., Gardner, W., Kazdin, A. E., & Vaught, R. S. (1980). Interrupted time-series analysis and its application to behavioral data. Journal of Applied Behavior Analysis, 13, 543–559.CrossRefPubMedPubMedCentralGoogle Scholar
  30. Huitema, B. E. (1985). Autocorrelation in applied behavior analysis: A myth. Behavioral Assessment, 109–120.Google Scholar
  31. Huitema, B. E. (1986). Autocorrelation in behavioral research: Wherefore art thou? In A. Poling & R. W. Fuqua (Eds.), Research methods in applied behavior analysis: Issues and advances (pp. 187–208). New York: Plenum Press.CrossRefGoogle Scholar
  32. Jones, R. R., Vaught, R. S., & Weinrott, M. (1977). Time-series analysis in operant research. Journal of Applied Behavior Analysis, 10, 151–166.CrossRefPubMedPubMedCentralGoogle Scholar
  33. Jones, R. R., Weinrott, M., & Vaught, R. S. (1978). Effects of serial dependency on the agreement between visual and statistical inference. Journal of Applied Behavior Analysis, 11, 277–283.CrossRefPubMedPubMedCentralGoogle Scholar
  34. Karp, H. J., & Fisch, G. S. (1996, May). Signal detection analysis of responses to graphic data produced by auto-correlation and least squares regression equations. Paper presented at the 22nd annual convention of the Association for Behavior Analysis, San Francisco.Google Scholar
  35. Kazdin, A. E. (1977). Artifact, bias, and complexity of assessment: The ABCs of reliability. Journal of Applied Behavior Analysis, 10, 141–150.CrossRefPubMedPubMedCentralGoogle Scholar
  36. Kazdin, A. E. (1978). Statistical analysis for single-case experimental designs. In M. Hersen & D. H. Barlow (Eds.), Single case experimental designs: Strategies for studying behavioral change (pp. 265–316). New York: Pergamon Press.Google Scholar
  37. Kelly, M. B. (1977). A review of the observational data-collection and reliability procedures reported in the Journal of Applied Behavior Analysis. Journal of Applied Behavior Analysis, 10, 97–101.CrossRefPubMedPubMedCentralGoogle Scholar
  38. Keselman, H. J., & Leventhal, L. (1974). Concerning the statistical procedures enumerated by Gentile et al.: Another perspective. Journal of Applied Behavior Analysis, 7, 643–645.CrossRefPubMedPubMedCentralGoogle Scholar
  39. Kratochwill, T., Alden, K., Demuth, D., Dawson, D., Panicucci, C., Arntson, P., McMurray, N., Hempstead, J., & Levin, J. (1974). A further consideration in the application of an analysis-of-variance model for the intrasubject replication design. Journal of Applied Behavior Analysis, 7, 629–633.CrossRefPubMedPubMedCentralGoogle Scholar
  40. Kratochwill, T. R., & Wetzel, R. J. (1977). Observer agreement, credibility, and judgment: Some considerations in presenting observer agreement data. Journal of Applied Behavior Analysis, 10, 133–139.CrossRefPubMedPubMedCentralGoogle Scholar
  41. Lewandowsky, S., & Spence, I. (1989). Discriminating strata in scatterplots. Journal of the American Statistical Association, 84, 682–688.CrossRefGoogle Scholar
  42. Magnusson, D., Bergman, L. R., Rudinger, G., & Törestad, B. (1991). Problems and methods in longitudinal research: Stability and change. Cambridge, England: Cambridge University Press.CrossRefGoogle Scholar
  43. Matyas, T. A., & Greenwood, K. M. (1990). Visual analysis of single-case time series: Effects of variability, serial dependence, and magnitude of intervention effects. Journal of Applied Behavior Analysis, 23, 341–351.CrossRefPubMedPubMedCentralGoogle Scholar
  44. Matyas, T. A., & Greenwood, K. M. (1991). Problems in the estimation of autocorrelation in brief time series and some implications for behavioral data. Behavioral Assessment, 13, 137–157.Google Scholar
  45. McEwan, J. S. A. (1994). Optimising line graph aspect ratio. Unpublished doctoral dissertation, The University of Waikato, New Zealand.Google Scholar
  46. Michael, J. (1974). Statistical inference for individual organism research: Mixed blessing or curse? Journal of Applied Behavior Analysis, 7, 647–653.CrossRefPubMedPubMedCentralGoogle Scholar
  47. Ottenbacher, K. J. (1986). Reliability and accuracy of visually analyzing graphed data from single-subject designs. American Journal of Occupational Therapy, 40, 464–469.CrossRefPubMedGoogle Scholar
  48. Parsonson, B. S., & Baer, D. M. (1992). The visual analysis of data, and current research into the stimuli controlling it. In T. R. Kratochwill & J. R. Levin (Eds.), Single-case research design and analysis (pp. 15–41). Hillsdale, NJ: Erlbaum.Google Scholar
  49. Pfadt, A., Cohen, I. L., Sudhalter, V., Romanczyk, R. G., & Wheeler, D. J. (1992). Applying statistical process control to clinical data: An illustration. Journal of Applied Behavior Analysis, 25, 551–560.CrossRefPubMedPubMedCentralGoogle Scholar
  50. Sharpley, C. F., & Alavosius, M. P. (1988). Autocorrelation in behavioral data: An alternative perspective. Behavioral Assessment, 10, 243–251.Google Scholar
  51. Shewhart, W. A. (1931). Economic control of quality of manufactured product. Princeton, NJ: Reinhold Co.Google Scholar
  52. Shrout, P. E. (1997). Should significance testing be banned? Introduction to a special section exploring the pros and cons. Psychological Science, 8, 1–2.CrossRefGoogle Scholar
  53. Sidman, M. (1952). A note on functional relations obtained from group data. Psychological Bulletin, 49, 263–269.CrossRefPubMedGoogle Scholar
  54. Sidman, M. (1960). Tactics of scientific research: Evaluation of experimental data in psychology. New York: Basic Books.Google Scholar
  55. Skinner, B. F. (1938). The behavior of organisms. New York: Appleton-Century-Crofts.Google Scholar
  56. Spence, I. (1990). Visual psychophysics of simple graphical elements. Journal of Experimental Psychology: Human Perception and Performance, 16, 683–692.PubMedGoogle Scholar
  57. Spence, I., & Lewandowsky, S. (1990). Graphical perception. In J. Fox & J. S. Long (Eds.), Modern methods of data analysis (pp. 13–57). Newbury Park, CA: Sage.Google Scholar
  58. Stigler, S. M. (1986). Estimating serial correlation by visual inspection of diagnostic plots. The American Statistician, 40, 111–116.Google Scholar
  59. Thoresen, C. E., & Elashoff, J. D. (1974). An analysis-of-variance model for intrasubject replication design: Some additional comments. Journal of Applied Behavior Analysis, 7, 639–641.CrossRefPubMedPubMedCentralGoogle Scholar
  60. Toothaker, L. E., Banz, M., Noble, C., Camp, J., & Davis, D. (1983). N = 1 designs: The failure of ANOVA-based tests. Journal of Educational Statistics, 8, 289–309.CrossRefGoogle Scholar
  61. Velicer, W. F., & Harrop, J. (1983). The reliability and accuracy of time series model identification. Evaluation Review, 7, 551–560.CrossRefGoogle Scholar
  62. Wampold, B., & Furlong, M. (1981). The heuristics of visual inference. Behavioral Assessment, 3, 79–92.Google Scholar
  63. Wetherill, G. B., & Brown, D. W. (1991). Statistical process control: Theory and practice. London: Chapman and Hall.CrossRefGoogle Scholar
  64. Yeltman, A. R., Wildman, B. G., & Erickson, M. T. (1977). A probability-based formula for calculating interobserver agreement. Journal of Applied Behavior Analysis, 127–131.Google Scholar

Copyright information

© Association for Behavior Analysis International 1998

Authors and Affiliations

  • Gene S. Fisch
    • 1
  1. 1.General Clinical Research CenterYale UniversityNew HavenUSA

Personalised recommendations