Skip to main content

Graph Construction and Visual Analysis: A Comparison of Curriculum-based Measurement Vendors

Journal of Behavioral Education volume 32, pages 90–108 (2023)Cite this article

Abstract

Curriculum-based measurement (CBM) represents a critical strategy for data-based decisionmaking within educational settings. Visual analysis is frequently used to analyze CBM data; thus, CBM vendors often automatically generate graphs based on student data to facilitate analysis. Differences in graph formatting are apparent across CBM vendors, and currently, the extent to which these differences may influence rater interpretation of data is unknown. As such, the current study sought to evaluate whether there were differences in rater evaluation of presence and magnitude of an intervention effect when identical data were plotted on graphs of four CBM vendors: AIMSweb, FastBridge Learning, easyCBM, and DIBELS Next. Results of the study indicated that visual analysts rated. Results of the study found that probability of identifying the presence of a treatment effect was similar across vendors. However, differences emerged with respect to ratings of magnitude of effect, with raters indicating the greatest magnitude of intervention effect for easyCBM and DIBELS Next graphs. As data graphs generated by CBM vendors may be utilized by school personnel to make decisions regarding the continuation, discontinuation, or alteration of intervention procedures, the results of the study indicate the need for increased consideration in the manner in which data are presented visually such that accurate decisions are made regarding student progress and response to intervention.

This is a preview of subscription content, access via your institution.

Access options

Buy single article

Instant access to the full article PDF.

USD 39.95

Price includes VAT (Canada)

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

References

  • Anderson, D., Alonzo, J., Tindal, G., Farley, D., Irvin, P. S., Lai, C., Saven, J. L., Wray, K. A. (2014). Technical Manual: Easycbm. Behavioral Research and Teaching, University of Oregon. Report #1408.

  • Ardoin, S. P., Christ, T. J., Morena, L. S., Cormier, D. C., & Klingbeil, D. A. (2013). A systematic review and summarization of the recommendations and research surrounding curriculum-based measurement of oral reading fluency (CBM-R) decision rules. Journal of School Psychology, 51, 1–18.

    Article  PubMed  Google Scholar 

  • Bates, D., Machler, M., Bolker, B., & Walker, S. (2015). Fitting linear mixed-effects models using lme4. Journal of Statistical Software, 67, 1–48.

    Article  Google Scholar 

  • Benson, N. F., Floyd, R. G., Kranzler, J. H., Eckert, T. L., Fefer, S. A., & Morgan, G. B. (2019). Test use and assessment practices of school psychologists in the United States: Findings from the 2017 National Survey. Journal of School Psychology, 72, 29–48.

    Article  PubMed  Google Scholar 

  • Christ, T. J., & Ardoin, S. P. (2009). Curriculum-based measurement of oral reading: Passage equivalence and probe set development. Journal of School Psychology, 47, 55–75.

    Article  Google Scholar 

  • Christ, T. J., Aranas, Y.A., Brown, R., Johnson, L., Kember, J. M., Kilgus, S., Kiss, A., … Windram, H. (2018). FastBridge learning: Benchmarks and norms for 2018–19. Minneapolis, MN: Author and FastBridge Learning, LLC.

  • Christ, T. J., Zopluoglu, C., Monaghen, B., & Van Norman, E. R. (2013). Curriculum-based measurement of oral reading: Multi-study evaluation of schedule, duration, and dataset quality on progress monitoring outcomes. Journal of School Psychology, 51, 19–57.

    Article  PubMed  Google Scholar 

  • Christ, T. J., & Silberglitt, B. (2007). Estimates of the standard error of measurement for curriculum-based measures of oral reading fluency. School Psychology Review, 36, 130–146.

    Article  Google Scholar 

  • Dart, E. H., & Radley, K. C. (2017). The impact of ordinate scaling on the visual analysis of single-case data. Journal of School Psychology, 63, 105–118.

    Article  PubMed  Google Scholar 

  • Dart, E. H., & Radley, K. C. (2018). Toward a standard assembly of linear graphs. School Psychology Quarterly, 33, 350–355.

    Article  PubMed  Google Scholar 

  • Deno, S. L. (1985). Curriculum-based measurement: The emerging alternative. Exceptional Children, 52, 219–232.

    Article  PubMed  Google Scholar 

  • Deno, S. L. (2003). Developments in curriculum-based measurement. The Journal of Special Education, 37, 184–192.

    Article  Google Scholar 

  • Good, R. H., III, Kaminski, R. K., Cummings, K., Dufour-Martel, C., Petersen, K., Powell-Smith, K., Stollar, S., & Wallin, J. (2011). Acadience reading assessment manual. Eugene, OR: Dynamic measurement group. Available: https://acadiencelearning.org/.

  • Horner, R. H., Carr, E. G., Halle, J., McGee, G., Odom, S., & Wolery, M. (2005). The use of single-subject research to identify evidence-based practices in special education. Exceptional Children, 71(2), 165–179.

    Article  Google Scholar 

  • Hintze, J. M., Wells, C. S., Marcotte, A. M., & Solomon, B. G. (2018). Decision-making accuracy of CBM progress-monitoring data. Journal of Psychoeducational Assessment, 36, 74–81.

    Article  Google Scholar 

  • Jenkins, J. R., Schiller, E., Blackorby, J., Thayer, S. K., & Tilly, W. D. (2013). Responsiveness to intervention in reading: Architecture and practices. Learning Disability Quarterly, 36, 36–46.

    Article  Google Scholar 

  • Kratochwill, T. R., Hitchcock, J., Horner, R. H., Levin, J. R., Odom, S. L., Rindskopf, D. M., & Shadish, W. R. (2010). Single case design technical documentation. https://eric.ed.gov/?id=ED510743.

  • Matyas, T. A., & Greenwood, K. M. (1990). Visual analysis of single-case time series: Effects of variability, serial dependence, and magnitude of intervention effects. Journal of Applied Behavior Analysis, 23, 341–351.

    Article  PubMed  PubMed Central  Google Scholar 

  • McIntosh, K., & Goodman, S. (2016). Integrated multi-tiered systems of support: Blending RTI and PBIS. Guilford Press.

  • Microsoft Corporation. (2020). Microsoft Excel. https://office.microsoft.com/excel.

  • Ninci, J., Vannest, K. J., Willson, V., & Zhang, N. (2015). Interrater agreement between visual analysts of single-case data: A meta-analysis. Behavior Modification, 39, 510–541.

    Article  PubMed  Google Scholar 

  • Nelson, P. M., Van Norman, E. R., & Christ, T. J. (2017). Visual analysis among novices: Training and trend lines as graphic aids. Contemporary School Psychology, 21, 93–102.

    Article  Google Scholar 

  • Parker, R. I., Vannest, K. J., Davis, J. L., & Clemens, N. H. (2012). Defensible progress monitoring data for medium-and high-stakes decisions. The Journal of Special Education, 46, 141–151.

    Article  Google Scholar 

  • R Core Team (2020). R: A language and environment for statistical computing. R Foundation for Statistical Computing. https://www.R-project.org.

  • Radley, K. C., Dart, E. H., & Wright, S. J. (2018). The effect of data points per x-to y-axis ratio on visual analysts evaluation of single-case graphs. School Psychology Quarterly, 33, 314–322.

    Article  PubMed  Google Scholar 

  • Reschly, A. L., Busch, T. W., Betts, J., Deno, S. L., & Long, J. D. (2009). Curriculum-based measurement oral reading as an indicator of reading achievement: A meta-analysis of the correlational evidence. Journal of School Psychology, 47, 427–469.

    Article  PubMed  Google Scholar 

  • Shinn, M. R., & Shinn, M. M. (2003). AimswebTM training workbook administration and scoring of reading curriculum-based measurement (r-cbm) for use in general outcome measurement. Edformation, Inc.

  • Stecker, P. M., Fuchs, D., & Fuchs, L. S. (2008a). Progress monitoring as essential practice within response to intervention. Rural Special Education Quarterly, 27, 10–17.

    Article  Google Scholar 

  • Stecker, P. M., Lembke, E. S., & Foegen, A. (2008b). Using progress-monitoring data to improve instructional decision making. Preventing School Failure: Alternative Education for Children and Youth, 52, 48–58.

    Article  Google Scholar 

  • Van Norman, E. R., Nelson, P. M., Shin, J. E., & Christ, T. J. (2013). An evaluation of the effects of graphic aids in improving decision accuracy in a continuous treatment design. Journal of Behavioral Education, 22(4), 283–301.

    Article  Google Scholar 

  • Van Norman, E. R., & Christ, T. J. (2016). How accurate are interpretations of curriculum-based measurement progress monitoring data? Visual analysis versus decision rules. Journal of School Psychology, 58, 41–55.

    Article  PubMed  Google Scholar 

  • Ximenes, V. M., Manolov, R., Solanas Pérez, A., & Quera, V. Q. J. (2009). Factors affecting visual analysis in single-case designs. Spanish Journal of Psychology, 12, 823–832.

    Article  PubMed  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. University of South Florida, Tampa, USA

    Evan H. Dart

  2. Lehigh University, Bethlehem, USA

    Ethan R. Van Norman

  3. University of Wisconsin – Madison, Madison, USA

    David A. Klingbeil

  4. University of Utah, Salt Lake City, USA

    Keith C. Radley

Authors
  1. Evan H. Dart
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ethan R. Van Norman
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. David A. Klingbeil
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Keith C. Radley
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Evan H. Dart.

Ethics declarations

Confict of interest

The authors have no conflicts of interest to report, financial or otherwise.

Ethical Approval

All procedures were approved by the first author’s University Institutional Review Board (IRB).

Informed consent

All subjects provided informed consent to participate.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Dart, E.H., Van Norman, E.R., Klingbeil, D.A. et al. Graph Construction and Visual Analysis: A Comparison of Curriculum-based Measurement Vendors. J Behav Educ 32, 90–108 (2023). https://doi.org/10.1007/s10864-021-09440-7

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10864-021-09440-7

Keywords

  • Visual analysis
  • Curriculum-based measurement
  • Graph construction