Skip to main content

Visual Analysis Among Novices: Training and Trend Lines as Graphic Aids

Abstract

The current study evaluated the degree to which novice visual analysts could discern trends in simulated time-series data across differing levels of variability and extreme values. Forty-five novice visual analysts were trained in general principles of visual analysis. One group received brief training on how to identify and omit extreme values. Participants rated 72 continuous time-series graphs. Inferential analyses were used to estimate the probability of correct responses. Participants who received the additional training were more likely to correctly identify intervention effects across all conditions. Nevertheless, extreme values had a substantial impact on decision accuracy for all participants. The impact of extreme values was exacerbated by increases in overall variability. Results support the notion that automated trend lines are useful but not infallible when interpreting continuous time-series data. Implications for practice and avenues for future research are discussed.

This is a preview of subscription content, access via your institution.

References

  • Akaike, H. (1974). A new look at the statistical model identification. Automatic Control, IEEE Transactions, 19, 716–723.

    Article  Google Scholar 

  • Ardoin, S. P., Christ, T. J., Morena, L. S., Cormier, D. C., & Klingbeil, D. A. (2013). A systematic review and summarization of recommendations and research surrounding Curriculum Based Measurement of Oral Reading Fluency (CBM-R) decision rules. Journal of School Psychology, 51. doi: 10.1016/j.jsp.2012.09.04

  • Begeny, J. C., & Martens, B. K. (2006). Assessing pre-service teachers’ training in empirically-validated behavioral instruction practices. School Psychology Quarterly, 21(3), 262–285.

    Article  Google Scholar 

  • Berkeley, S., Bender, W. N., Peaster, L. G., & Saunders, L. (2009). Implementation of response to intervention: a snapshot of progress. Journal of Learning Disabilities, 42(1), 85–95.

    Article  PubMed  Google Scholar 

  • Bergan, J. R., & Kratochwill, T. R. (1990). Behavioral consultation. Columbus, OH: Merrill.

    Google Scholar 

  • Christ, T. J. (2006). Short term estimates of growth using curriculum-based measurement of oral reading fluency: estimates of standard error of the slope to construct confidence intervals. School Psychology Review, 35(1), 128–133.

    Google Scholar 

  • Christ, T. J., Nelson, P. M., Van Norman, E. R., Chafouleas, S. M., & Riley-Tillman, T. C. (2014). Direct behavior rating: an evaluation of time-series interpretations as consequential validity. School Psychology Quarterly, 29, 157–170.

  • Deno, S. L. (2003). Developments in curriculum-based measurement. The Journal of Special Education, 37(3), 184–192. doi:10.1177/00224669030370030801.

    Article  Google Scholar 

  • Deno, S. L. (1990). Individual differences and individual difference the essential difference of special education. The Journal of Special Education, 24(2), 160–173. doi:10.1177/002246699002400205.

    Article  Google Scholar 

  • Deno, S. L., Fuchs, L. S., Marston, D., & Shin, J. (2001). Using curriculum-based measurements to establish growth standards for students with learning disabilities. School Psychology Review, 30, 507–524.

  • Bates, D., Maechler, M., & Bolker, B. (2012). lme4: linear mixed-effects models using S4 classes. R package version 0.999999–0. http://CRAN.R-project.org/package=lme4

  • Fuchs, L. S., & Fuchs, D. (1986). Effects of systematic formative evaluation: a meta-analysis. Exceptional Children, 53, 199–208.

    PubMed  Google Scholar 

  • Fuchs, L. S., Fuchs, D., Hamlett, C. L., & Ferguson, C. (1992). Effects of expert system consultation within curriculum-based measurement using a reading maze task. Exceptional Children, 58, 436–450.

    Google Scholar 

  • Gibson, G., & Ottenbacher, K. (1988). Characteristics influencing the visual analysis of single-subject data: an empirical analysis. The Journal of Applied Behavior Science, 24, 298–314.

    Article  Google Scholar 

  • Graney, S. B. (2008). General education teacher judgments of their low-performing students’ short-term reading progress. Psychology in the Schools, 45(6), 537–549.

    Article  Google Scholar 

  • Hojem, M. A., & Ottenbacher, K. J. (1988). Empirical investigation of visual-inspection versus trend-line analysis of single-subject data. Physical Therapy, 68(6), 983–988.

    Article  PubMed  Google Scholar 

  • Huitema, B. E. (1986). Autocorrelation in behavioral research: wherefore art thou? In A

  • Matyas, T. A., & Greenwood, K. M. (1990). Visual analysis of single-case time series: effects of variability, serial dependence, and magnitude of intervention effects. Journal of Applied Behavior Analysis, 23(3), 341–351. doi:10.1901/jaba.1990.23-341.

    Article  PubMed  PubMed Central  Google Scholar 

  • Ninci, J., Vannest, K. J., Wilson, V., & Zhang, N. (2015). Interrater agreement between visual analysts of single-case data: a meta-analysis. Behavior Modification, 39, 510–541. doi:10.1177/0145445515581327.

    Article  PubMed  Google Scholar 

  • Normand, M. P., & Bailey, J. S. (2006). The effects of celebration lines of visual data analysis. Behavior Modification, 30, 295–314.

    Article  PubMed  Google Scholar 

  • Ottenbacher, K. J., & Cusick, A. (1991). An empirical investigation of interrater agreement for single-subject data using graphs with and without trend lines. Journal of the Association for Persons with Severe Handicaps, 16(1), 48–55.

  • Ottenbacher, K. J. (1990). Visual inspection of single-subject data: an empirical analysis. Mental Retardation, 28, 283–290.

    PubMed  Google Scholar 

  • Parker, R. I., Cryer, J., & Byrns, G. (2006). Controlling baseline trend in single-case research. School Psychology Quarterly, 21, 418–443.

    Article  Google Scholar 

  • Prewett, S., Mellard, D. F., Deshler, D. D., Allen, J., Alexander, R., & Stern, A. (2012). Response to intervention in middle schools: practices and outcomes. Learning Disabilities Research & Practice, 27(3), 136–147.

    Article  Google Scholar 

  • Shadish, W. R., Cook, T. D., & Campbell, D. T. (2002). Experimental and quasi-experimental designs for generalized causal inference. Belmont, CA: Wadsworth.

    Google Scholar 

  • Shapiro, E. S. (2004). Academic skills problems: direct assessment and intervention (3rd ed.). New York: Guilford Press.

    Google Scholar 

  • Stecker, P. M., & Fuchs, L. S. (2000). Effecting superior achievement using curriculum-based measurement: the importance of individual progress monitoring. Learning Disabilities Research & Practice, 15(3), 128–134.

    Article  Google Scholar 

  • Skiba, R., Deno, S., Marston, D., & Casey, A. (1989). Influence of trend estimation and subject familiarity on practitioners’ judgments of intervention effectiveness. The Journal of Special Education, 22, 433–445.

    Article  Google Scholar 

  • Tilly III, W. D. (2008). The evolution of school psychology to science-based practice: problem solving and the three-tiered model. In A. Thomas & J. Grimes (Eds.), Best practices in school psychology V (pp. 17–36). Bethesda, MD: National Association of School Psychologists.

    Google Scholar 

  • Wolery, M., & Harris, S. R. (1982). Interpreting results of single-subject research design. Journal of the American Physical Therapy Association, 62, 445–452.

    Google Scholar 

  • Van Norman, E. R., Nelson, P. M., Shin, J. E., & Christ, T. J. (2013). An evaluation of the effects of graphic aids in improving decision accuracy in a continuous treatment design. Journal of Behavioral Education, 1–19. doi: 10.1007/s10864-013-9176-2

  • Ximenes, V. M., Manolov, R., Solanas, A., & Quera, V. (2009). Factors affecting visual inference in single-case designs. The Spanish Journal of Psychology, 12, 823–832.

    Article  PubMed  Google Scholar 

  • Zieffler, A. S., & Garfield, J. B. (2009). Modeling the growth of students’ covariational reasoning during an introductory statistics course. Statistics Education Research Journal, 8(1), 7–31.

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Peter M. Nelson.

Electronic supplementary material

ESM 1

(DOCX 78 kb)

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Nelson, P.M., Van Norman, E.R. & Christ, T.J. Visual Analysis Among Novices: Training and Trend Lines as Graphic Aids. Contemp School Psychol 21, 93–102 (2017). https://doi.org/10.1007/s40688-016-0107-9

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s40688-016-0107-9

Keywords

  • Visual analysis
  • Progress monitoring
  • Data-based decision-making