Skip to main content

Advertisement

SpringerLink
Log in

Investigating the quality of the school technology needs assessment (STNA) 3.0: A validity and reliability study

  • Research
  • Published:
Educational Technology Research and Development Aims and scope Submit manuscript

Abstract

Schools and districts should use a well-designed needs assessment to inform important decisions about a range of technology program areas. Presently, there is a lack of valid and reliable instruments available and accessible to schools to effectively assess their educational needs to better design and evaluate their projects and initiatives. The School Technology Needs Assessment (STNA) is a free, user-friendly online survey tool that meets this need for planning and formative evaluation of technology projects in educational settings. This study used existing data from a robust sample (n = 1918) of educators from across North Carolina to examine the reliability and validity of STNA. A collective review of study results including the literature review, exploratory factor analysis (EFA), confirmatory factor analysis (CFA), and internal consistently reliability analysis indicated that STNA was a high-quality instrument.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1

Similar content being viewed by others

References

  • Apple Computer Inc. (1995). Changing the conversation about teaching, learning, and technology: A report on 10 years of ACOT research. Cupertino, CA: Apple Computer, Inc.

    Google Scholar 

  • Bangert-Drowns, R. L., & Pyke, C. (2002). Teacher ratings of student engagement with educational software: An exploratory study. Educational Technology Research and Development, 50(2), 23.

    Article  Google Scholar 

  • Baylor, A. L., & Ritchie, D. (2002). What factors facilitate teacher skill, teacher morale, and perceived student learning in technology-using classrooms? Computers and Education, 39(4), 395–414.

    Article  Google Scholar 

  • Bebell, D., Russell, M., & O’Dwyer, L. (2004). Measuring teachers’ technology uses: Why multiple-measures are more revealing. Journal of Research on Technology in Education, 37(1), 45–63.

    Google Scholar 

  • Becker, H., Ravitz, J., & Wong, Y. (1999). Teacher and teacher-directed student use of computers and software. Irvine, CA: Center for Research on Information Technology and Organizations.

    Google Scholar 

  • Boethel, M., & Dimock, K. V. (1999). Constructing knowledge with technology. Austin, TX: Southwest Educational Development Laboratory.

    Google Scholar 

  • Bogdan, R., & Biklen, S. (2007). Qualitative research for education (5th ed.). Boston: Pearson Education, Inc.

    Google Scholar 

  • Bollen, K. A. (1986). Sample size and Bentler and Bonett’s nonnormed fit index. Psychometrika, 51, 375–377.

    Article  Google Scholar 

  • Bradshaw, L. K. (2002). Technology for teaching and learning: Strategies for staff development and follow-up support. Journal of Technology and Teacher Education, 10(1), 131–150.

    Google Scholar 

  • Brush, T., Armstrong, J., Barbrow, D., & Ulintz, L. (1999). Design and delivery of integrated learning systems: Their impact on student achievement and attitudes. Journal of Educational Computing Research, 21(4), 475–486.

    Google Scholar 

  • Byrne, B. (2001). Structural equation modeling with AMOS: Basic concepts, applications, and programming. Mahway, NJ: Lawrence Erlbaum Associates, Inc.

    Google Scholar 

  • CEO Forum. (2000a). School technology and readiness report. Washington, DC: CEO Forum.

    Google Scholar 

  • CEO Forum. (2000b). Teacher preparation STaR Chart: A self-assessment tool for colleges of education. Washington, DC: CEO Forum on Education and Technology.

    Google Scholar 

  • CEO Forum. (2001). Key building blocks for student achievement in the 21st century. Washington, DC: CEO Forum.

    Google Scholar 

  • Christensen, R., & Knezek, G. (1996). Constructing the teachers’ attitudes toward computers (TAC) questionnaire. Paper presented at the annual meeting of the southwest educational research association, New Orleans, LA.

  • Christensen, R., & Knezek, G. (2000). Internal consistency reliabilities for 14 computer attitude scales. Journal of Technology and Teacher Education, 7(4), 327–336.

    Google Scholar 

  • Christmann, E., & Badgett, J. (1999). A comparative analysis of the effects of computer-assisted instruction on student achievement in differing science and demographic areas. Journal of Computers in Mathematics and Science Teaching, 18(2), 135–143.

    Google Scholar 

  • Clements, D. H., & Sarama, J. (2003). Strip mining for gold: Research and policy in educational technology—a response to “Fool’s Gold”. Educational Technology Review, 11(1), 7–69.

    Google Scholar 

  • Cohen, D., & Ball, D. L. (1999). Instruction, capacity, and improvement (CPRE research report series RR-43). Philadelphia, PA: University of Pennsylvania Consortium for Policy Research in Education.

    Google Scholar 

  • Cohen, D., & Hill, H. (1998). Instructional policy and classroom performance: The mathematics reform in California. Philadelphia, PA: Consortium for Policy Research in Education.

    Google Scholar 

  • Cook, T., & Campbell, D. (1979). Quasi-experimentation design and analysis issues for field settings. Boston: Houghton Mifflin.

    Google Scholar 

  • Cook, C., Heath, F., & Thompson, R. (2000). A meta-analysis of response rates in web or Internet-based surveys. Educational and Psychological Measurement, 60, 821–836.

    Article  Google Scholar 

  • Cordray, D., & Pion, G. (2006). Treatment strength and integrity: Models and methods. In R. R. Bootzin & P. E. McKnight (Eds.), Strengthening research methodology: Psychological measurement and evaluation (pp. 103–124). Washington, DC: American Psychological Association Press.

    Chapter  Google Scholar 

  • Cotton, K. (1991). Computer-assisted instruction. Retrieved December 18, 2006, from http://www.nwrel.org/scpd/sirs/5/cu10.html

  • Culp, K., Honey, M., & Mandinach, E. (2003). A retrospective on twenty years of education technology policy. Newton, MA: Education Development Center.

    Google Scholar 

  • Darling-Hammond, L., & McLaughlin, M. (1996). Policies that support professional development in an era of reform. In M. McLaughlin & I. Oberman (Eds.), Teacher learning: New policies, new practices (pp. 202–218). New York: Teachers College Press.

    Google Scholar 

  • Davis, S., Darling-Hammond, L., LaPointe, M., & Meyerson, D. (2005). School leadership study: Developing successful principals. Stanford, CA: Stanford Educational Leadership Institute.

    Google Scholar 

  • Dick, W., Carey, L., & Carey, J. (2005). Systematic design of instruction (6th ed.). Boston: Pearson.

    Google Scholar 

  • Epstein, J. (2001). School, family, and community partnerships: Preparing educators and improving schools. Boulder, CO: Westview Press.

    Google Scholar 

  • Ertmer, P. A., Gopalakrishnan, S., & Ross, E. M. (2001). Technology-using teachers: Comparing perceptions of exemplary technology use to best practice. Journal of Research on Technology in Education, 33(5).

  • Fraenkel, J., & Wallen, N. (2003). How to design and evaluate research in education (5th ed.). Boston: McGraw Hill.

    Google Scholar 

  • Fuchs, L., & Fuchs, D. (2002). Curriculum-based measurement: Describing competence, enhancing outcomes, evaluating treatment effects, and identifying treatment nonresponders. Peabody Journal of Education, 77(2), 64–84.

    Article  Google Scholar 

  • Fullan, M. (2003). The moral imperative of school leadership. Thousand Oaks, CA: Corwin Press.

    Google Scholar 

  • Gable, R., & Wolf, M. (1993). Instrument development in the affective domain (2nd ed.). Boston: Kluwer Academic Publishers.

    Google Scholar 

  • Gall, M., Borg, W., & Gall, J. (1996). Educational research (6th ed.). White Plains, NY: Longman.

    Google Scholar 

  • Garet, M., Porter, A., Desimone, L., Birman, B., & Yoon, K. (2001). What makes professional development effective? Results from a national sample of teachers. American Educational Research Journal, 38(4), 915–945.

    Article  Google Scholar 

  • Guha, S. (2001). Integrating computers in elementary grade classroom instruction: Analyses of teachers’ perceptions in present and preferred situations. Journal of Educational Computing Research, 24(3), 275–303.

    Article  Google Scholar 

  • Guskey, T. (2000). Evaluating professional development. Thousand Oaks, CA: Corwin Press, Inc.

    Google Scholar 

  • Haertel, G., & Means, B. (2000). Stronger designs for research on educational uses of technology: Conclusions and implications. Menlo Park, CA: SRI International.

    Google Scholar 

  • Hofer, M., Chamberlin, B., & Scot, T. (2004). Fulfilling the need for a technology integration specialist. T.H.E. Journal, 32(3), 34.

    Google Scholar 

  • Hu, L., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling, 6(1), 1–55.

    Article  Google Scholar 

  • International Society for Technology in Education. (2000a). National educational technology standards for students. OR: Eugene.

    Google Scholar 

  • International Society for Technology in Education. (2000b). National educational technology standards for teachers. Eugene, OR: International Society for Technology in Education.

    Google Scholar 

  • International Society for Technology in Education. (2002). National educational technology standards for administrators. Eugene, OR: International Society for Technology in Education.

    Google Scholar 

  • Interstate School Leaders Licensure Consortium. (1996). Standards for school leaders. Washington, DC: Council of Chief State School Officers.

    Google Scholar 

  • Jonassen, D. (1999). Designing constructivist learning environments. In C. M. Reigeluth (Ed.), Instructional design theories and models (Vol. II, pp. 215–239). Mahwah, NJ: Lawrence Erlbaum Associates.

    Google Scholar 

  • Joyce, B., & Showers, B. (1995). Student achievement through staff development. White Plains, NY: Longman.

    Google Scholar 

  • Kaiser, H. F. (1960). The application of electronic computers to factor analysis. Educational and Psychological Measurement, 20, 141–151.

    Article  Google Scholar 

  • Kanaya, T., Light, D., & Culp, K. (2005). Factors influencing outcomes from a technology-focused professional development program. Journal of Research on Technology in Education, 37(2), 313–329.

    Google Scholar 

  • Kerrigan, J. (2002). Powerful software to enhance the elementary school mathematics program. Teaching Children Mathematics, 8(6), 364–370.

    Google Scholar 

  • Killion, J. (2002). Assessing impact: Evaluating staff development. Oxford, OH: National Staff Development Council.

    Google Scholar 

  • Knestis, K., Byrom, E., Matzen, N., Corn, J., & Thrift, B. (2007). Factors influencing the capacity for applying project evaluation in schools: Lessons learned from looking at North Carolina education technology (LANCET). Greensboro, NC: SERVE Center at UNCG. Available from http://www.serve.org/Evaluation/Capacity/Lessons/LANCETLessonsLearned.pdf.

  • Kocher, A., & Moore, B. (2001, April). Assessing teacher technology skills. Paper presented at the annual meeting of the American educational research association, Seattle, WA.

  • Lambert, L. (2002). A framework for shared leadership. Educational Leadership, 3, 7–40.

    Google Scholar 

  • Land, S., & Hannafin, M. (2000). Student-centered learning environments. In D. Jonassen & S. Land (Eds.), Theoretical foundations of learning environments. Mahwah, NJ: Lawrence Erlbaum Associates.

    Google Scholar 

  • Lemke, C., Wainer, A., & Haning, N. (2006). National trends: Enhancing education through technology, NCLB Title II D—year three in review. Los Angeles: Metiri Group.

    Google Scholar 

  • Linn, R., & Miller, M. (2005). Measurement and assessment in teaching (9th ed.). Upper Saddle River, NJ: Pearson Education, Inc.

    Google Scholar 

  • Loucks-Horsely, S., Love, N., Stiles, K., Mundry, S., & Hewson, P. (2003). Designing professional development for teachers of science and mathematics (2nd ed.). Thousand Oaks, CA: Corwin Press.

    Google Scholar 

  • Lowther, D. L., & Ross, S. M. (2000). Teacher technology questionnaire (TTQ). Memphis, TN: Center for Research in Educational Policy, The University of Memphis.

    Google Scholar 

  • Marzano, R., Waters, T., & McNulty, B. (2005). School leadership that works. Aurora, CO: McREL.

    Google Scholar 

  • McKenzie, J. (2002). The true cost of ownership. MultiMedia Schools, 9(6), 24–28.

    Google Scholar 

  • Meredith, W. (1993). Measurement invariance, factor analysis, and factorial invariance. Psychometrika, 58, 525–543.

    Article  Google Scholar 

  • Merriam, S., Caffarella, R., & Baumgartner, L. (2007). Learning in adulthood (3rd ed.). San Francisco: Jossey Bass.

    Google Scholar 

  • Milken Exchange. (1999). Transforming learning through technology: Policy roadmaps for the nation’s governors. Santa Monica, CA: Milken Family Foundation.

    Google Scholar 

  • Miller, M. (1995). Coefficient alpha: A basic introduction from the perspectives of classical test theory and structural equation modeling. Structural Equation Modeling, 2(3), 255–273.

    Article  Google Scholar 

  • National Business Education Alliance. (2006). The LoTi connection. Retrieved June 6, 2007, from http://www.loticonnection.com/

  • National Staff Development Council. (2001). Standards of staff development (Rev. Edn.). from http://www.nsdc.org/standards/index.cfm

  • NCRTEC. (1997). Learning with technology profile tool. Retrieved June 30, 2007, from http://www.ncrtec.org/capacity/profile/profwww.htm

  • Niederhauser, D., & Stoddart, T. (2001). Teachers’ instructional perspectives and use of educational software. Teaching and Teacher Education, 17, 15–31.

    Article  Google Scholar 

  • North Carolina Department of Public Instruction. (2003). Looking at North Carolina’s educational technology (LANCET). from http://www.ncwiseowl.org/impact/LANCET/

  • North Carolina Department of Public Instruction. (2007). Annual media and technology report. Retrieved December, 20, 2007, from http://tps.dpi.state.nc.us/amtr2007/

  • North Central Regional Educational Laboratory. (2000). enGauge: A framework for effective technology use. Retrieved June 4, 2007, from http://www.ncrel.org/engauge/

  • November, A., Costello, M., & Huske, L. (1998). Developing a school or district technology plan. North central regional educational laboratory pathways critical issue.

  • Office of Technology Assessment. (1995). Teachers and technology: Making the connection. Washington, DC: U.S. Government Printing Office.

    Google Scholar 

  • Otterbourg, S. (1998). Using technology to strengthen employee and family involvement in education. Washington, DC: Partnership for Family Involvement in Education; U.S. Department of Education.

    Google Scholar 

  • Papanastasiou, E., Zemblyas, M., & Vrasidas, C. (2003). Can computer use hurt science achievement? Journal of Science Education and Technology, 12(3), 325–332.

    Article  Google Scholar 

  • Partnership for 21st Century Skills. (2006). Framework for 21st century learning overview. Retrieved November 1, 2006, from http://www.21stcenturyskills.org/documents/Frameworkflyer092806.pdf

  • Pendersen, S., & Liu, M. (2003). Teachers’ beliefs about issues in the implementation of a student-centered learning environment. Educational Technology Research and Development, 51(2), 57–76.

    Article  Google Scholar 

  • Penuel, W., Kim, D., Michalchik, V., Lewis, S., Means, B., Murhpy, R., et al. (2002). Using technology to enhance connections between home and school: A research synthesis. Washington, DC: Planning and Evaluation Service, U.S. Department of Education.

    Google Scholar 

  • Pett, M., Lackey, N., & Sullivan, J. (2003). Making sense of factor analysis: The use of factor analysis for instrument development in health care research. Thousand Oaks, CA: Sage.

    Google Scholar 

  • Powell, J., Aeby, V., & Carpenter-Aeby, T. (2003). A comparison of student outcomes with and without teacher facilitated computer-based instruction. Computers and Education, 40(2), 183–191.

    Article  Google Scholar 

  • Public Schools of North Carolina. (2005). IMPACT: Guidelines for North Carolina media and technology programs. NC: Raleigh.

    Google Scholar 

  • Reilly, R. (1999). The technology coordinator: Curriculum leader or electronic janitor? MultiMedia Schools, 6(3), 38–41.

    Google Scholar 

  • Ringstaff, C., & Kelley, L. (2002). The learning return on our educational technology investment. San Francisco: WestEd.

    Google Scholar 

  • Sands, S., & Buchholz, E. S. (1997). The underutilization of computers to assist in the remediation of dyslexia. International Journal of Instructional Media, 24(2), 153–175.

    Google Scholar 

  • Schacter, J. (1999). The impact of education technology on student achievement. Santa Monica, CA: Milken Exchange on Education Technology.

    Google Scholar 

  • Segars, A. H., & Grover, V. (1993). Re-Examining perceived ease of use and usefulness: A confirmatory factor analysis. MIS Quarterly, 17(4), 517–525.

    Article  Google Scholar 

  • Shadish, W., Cook, T., & Campbell, D. (2002). Experimental and quasi-experimental designs for generalized causal inference. Boston: Houghton Mifflin Company.

    Google Scholar 

  • Shannon, D. M., Johnson, T. E., Searcy, S., & Lott, A. (2002). Using electronic surveys: Advice from survey professionals. Practical Assessment, Research & Evaluation, 8(1). Retrieved from http://PAREonline.net/getvn.asp?v=8&n=1.

  • Shelly, G., Cashman, T., Gunter, R., & Gunter, G. (2002). Integrating technology into the classroom (2nd ed.). Boston: Thompson Learning.

    Google Scholar 

  • Speck, M., & Knipe, C. (2001). Why can’t we get it right? Professional development in our schools. Thousand Oaks, CA: Corwin Press, Inc.

  • State Educational Technology Directors Association. (2004). SETDA examples of data-based decision making. Retrieved June 21, 2007, from http://setda.org/web/guest/datadrivendecisionmaking

  • Stein, S., & Gewirtzman, L. (2003). Principal training on the ground: Ensuring highly qualified leadership. Portsmouth, NH: Heinemann.

    Google Scholar 

  • Texas Education Agency. (2001). Texas STaR Chart: A tool for planning and assessing school technology and readiness aligned with the Texas long-range plan for technology. Austin, TX: Author.

    Google Scholar 

  • Texas Education Agency. (2002). Texas STaR chart. Retrieved June 4, 2007, from http://starchart.esc12.net/default.html

  • Texas Education Agency. (2007). Texas STaR chart revised. Retrieved August 10, 2009, from http://starchart.esc12.net/default.html.

  • Thompson, B. (2004). Exploratory and confirmatory factor analysis: Understanding concepts and applications. Washington, DC: American Psychological Association.

    Book  Google Scholar 

  • Thompson, B., & Daniel, L. G. (1996). The analytic evidence for the construct validity of scores: A historical overview and some guidelines. Educational and Psychological Measurement, 56(2), 213–224.

    Article  Google Scholar 

  • Tiene, D., & Luft, P. (2001). Teaching in a technology-rich classroom. Educational Technology, 41(4), 21–31.

    Google Scholar 

  • U.S. Department of Education. (2002). Enhancing education through technology.

  • U.S. Department of Education. (2004). Toward a new gold age in American education: National education technology plan. Washington, DC: U.S. Department of Education.

    Google Scholar 

  • Utah Education Network (1999). Utah technology awareness project rubrics. Retrieved June 16, 2007, from http://www.uen.org/cgi-bin/websql/utahlink/UTAPdomains.htm

Download references

Author information

Authors and Affiliations

  1. North Carolina State University, Raleigh, NC, USA

    Jenifer O. Corn

Authors
  1. Jenifer O. Corn
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jenifer O. Corn.

Appendix

Appendix

See Tables 8, 9, and 10.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Corn, J.O. Investigating the quality of the school technology needs assessment (STNA) 3.0: A validity and reliability study. Education Tech Research Dev 58, 353–376 (2010). https://doi.org/10.1007/s11423-009-9140-y

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11423-009-9140-y

Keywords

Search

Navigation