Abstract
The current study evaluated the degree to which novice visual analysts could discern trends in simulated time-series data across differing levels of variability and extreme values. Forty-five novice visual analysts were trained in general principles of visual analysis. One group received brief training on how to identify and omit extreme values. Participants rated 72 continuous time-series graphs. Inferential analyses were used to estimate the probability of correct responses. Participants who received the additional training were more likely to correctly identify intervention effects across all conditions. Nevertheless, extreme values had a substantial impact on decision accuracy for all participants. The impact of extreme values was exacerbated by increases in overall variability. Results support the notion that automated trend lines are useful but not infallible when interpreting continuous time-series data. Implications for practice and avenues for future research are discussed.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Akaike, H. (1974). A new look at the statistical model identification. Automatic Control, IEEE Transactions, 19, 716–723.
Ardoin, S. P., Christ, T. J., Morena, L. S., Cormier, D. C., & Klingbeil, D. A. (2013). A systematic review and summarization of recommendations and research surrounding Curriculum Based Measurement of Oral Reading Fluency (CBM-R) decision rules. Journal of School Psychology, 51. doi: 10.1016/j.jsp.2012.09.04
Begeny, J. C., & Martens, B. K. (2006). Assessing pre-service teachers’ training in empirically-validated behavioral instruction practices. School Psychology Quarterly, 21(3), 262–285.
Berkeley, S., Bender, W. N., Peaster, L. G., & Saunders, L. (2009). Implementation of response to intervention: a snapshot of progress. Journal of Learning Disabilities, 42(1), 85–95.
Bergan, J. R., & Kratochwill, T. R. (1990). Behavioral consultation. Columbus, OH: Merrill.
Christ, T. J. (2006). Short term estimates of growth using curriculum-based measurement of oral reading fluency: estimates of standard error of the slope to construct confidence intervals. School Psychology Review, 35(1), 128–133.
Christ, T. J., Nelson, P. M., Van Norman, E. R., Chafouleas, S. M., & Riley-Tillman, T. C. (2014). Direct behavior rating: an evaluation of time-series interpretations as consequential validity. School Psychology Quarterly, 29, 157–170.
Deno, S. L. (2003). Developments in curriculum-based measurement. The Journal of Special Education, 37(3), 184–192. doi:10.1177/00224669030370030801.
Deno, S. L. (1990). Individual differences and individual difference the essential difference of special education. The Journal of Special Education, 24(2), 160–173. doi:10.1177/002246699002400205.
Deno, S. L., Fuchs, L. S., Marston, D., & Shin, J. (2001). Using curriculum-based measurements to establish growth standards for students with learning disabilities. School Psychology Review, 30, 507–524.
Bates, D., Maechler, M., & Bolker, B. (2012). lme4: linear mixed-effects models using S4 classes. R package version 0.999999–0. http://CRAN.R-project.org/package=lme4
Fuchs, L. S., & Fuchs, D. (1986). Effects of systematic formative evaluation: a meta-analysis. Exceptional Children, 53, 199–208.
Fuchs, L. S., Fuchs, D., Hamlett, C. L., & Ferguson, C. (1992). Effects of expert system consultation within curriculum-based measurement using a reading maze task. Exceptional Children, 58, 436–450.
Gibson, G., & Ottenbacher, K. (1988). Characteristics influencing the visual analysis of single-subject data: an empirical analysis. The Journal of Applied Behavior Science, 24, 298–314.
Graney, S. B. (2008). General education teacher judgments of their low-performing students’ short-term reading progress. Psychology in the Schools, 45(6), 537–549.
Hojem, M. A., & Ottenbacher, K. J. (1988). Empirical investigation of visual-inspection versus trend-line analysis of single-subject data. Physical Therapy, 68(6), 983–988.
Huitema, B. E. (1986). Autocorrelation in behavioral research: wherefore art thou? In A
Matyas, T. A., & Greenwood, K. M. (1990). Visual analysis of single-case time series: effects of variability, serial dependence, and magnitude of intervention effects. Journal of Applied Behavior Analysis, 23(3), 341–351. doi:10.1901/jaba.1990.23-341.
Ninci, J., Vannest, K. J., Wilson, V., & Zhang, N. (2015). Interrater agreement between visual analysts of single-case data: a meta-analysis. Behavior Modification, 39, 510–541. doi:10.1177/0145445515581327.
Normand, M. P., & Bailey, J. S. (2006). The effects of celebration lines of visual data analysis. Behavior Modification, 30, 295–314.
Ottenbacher, K. J., & Cusick, A. (1991). An empirical investigation of interrater agreement for single-subject data using graphs with and without trend lines. Journal of the Association for Persons with Severe Handicaps, 16(1), 48–55.
Ottenbacher, K. J. (1990). Visual inspection of single-subject data: an empirical analysis. Mental Retardation, 28, 283–290.
Parker, R. I., Cryer, J., & Byrns, G. (2006). Controlling baseline trend in single-case research. School Psychology Quarterly, 21, 418–443.
Prewett, S., Mellard, D. F., Deshler, D. D., Allen, J., Alexander, R., & Stern, A. (2012). Response to intervention in middle schools: practices and outcomes. Learning Disabilities Research & Practice, 27(3), 136–147.
Shadish, W. R., Cook, T. D., & Campbell, D. T. (2002). Experimental and quasi-experimental designs for generalized causal inference. Belmont, CA: Wadsworth.
Shapiro, E. S. (2004). Academic skills problems: direct assessment and intervention (3rd ed.). New York: Guilford Press.
Stecker, P. M., & Fuchs, L. S. (2000). Effecting superior achievement using curriculum-based measurement: the importance of individual progress monitoring. Learning Disabilities Research & Practice, 15(3), 128–134.
Skiba, R., Deno, S., Marston, D., & Casey, A. (1989). Influence of trend estimation and subject familiarity on practitioners’ judgments of intervention effectiveness. The Journal of Special Education, 22, 433–445.
Tilly III, W. D. (2008). The evolution of school psychology to science-based practice: problem solving and the three-tiered model. In A. Thomas & J. Grimes (Eds.), Best practices in school psychology V (pp. 17–36). Bethesda, MD: National Association of School Psychologists.
Wolery, M., & Harris, S. R. (1982). Interpreting results of single-subject research design. Journal of the American Physical Therapy Association, 62, 445–452.
Van Norman, E. R., Nelson, P. M., Shin, J. E., & Christ, T. J. (2013). An evaluation of the effects of graphic aids in improving decision accuracy in a continuous treatment design. Journal of Behavioral Education, 1–19. doi: 10.1007/s10864-013-9176-2
Ximenes, V. M., Manolov, R., Solanas, A., & Quera, V. (2009). Factors affecting visual inference in single-case designs. The Spanish Journal of Psychology, 12, 823–832.
Zieffler, A. S., & Garfield, J. B. (2009). Modeling the growth of students’ covariational reasoning during an introductory statistics course. Statistics Education Research Journal, 8(1), 7–31.
Author information
Authors and Affiliations
Corresponding author
Electronic supplementary material
ESM 1
(DOCX 78 kb)
Rights and permissions
About this article
Cite this article
Nelson, P.M., Van Norman, E.R. & Christ, T.J. Visual Analysis Among Novices: Training and Trend Lines as Graphic Aids. Contemp School Psychol 21, 93–102 (2017). https://doi.org/10.1007/s40688-016-0107-9
Published:
Issue Date:
DOI: https://doi.org/10.1007/s40688-016-0107-9