Skip to main content

Advertisement

SpringerLink
Log in

Technological literacy for students aged 6–18: a new method for holistic measuring of knowledge, capabilities, critical thinking and decision-making

  • Published:
International Journal of Technology and Design Education Aims and scope Submit manuscript

Abstract

Technological literacy is identified as a vital achievement of technology- and engineering-intensive education. It guides the design of technology and technical components of educational systems and defines competitive employment in technological society. Existing methods for measuring technological literacy are incomplete or complicated, unreliable, unstable and imprecise, time-consuming, and require large expenditures on resources. This paper presents a new method for valid and reliable measuring of technological literacy. The method encompasses three main components—knowledge, capabilities, and critical thinking and decision-making. It is centred on the standards for technological literacy issued by the International Technology and Engineering Educators Association. It has three key features. (1) A construct-measure-result front-ended approach, where a construct consists of an object, attribute, and entity; which causes reduction of measure-induced distortion and error. (2) A broad test range definition that provides stable and accurate measuring of technological literacy for 6–18-year-old students. (3) A genuine design approach including a multiple choice test item form determination consisting of content, criterion and construct validity, item discrimination, difficulty index, and an intraclass correlation measure for time stability and scooping its heterogeneous nature. Only the method is described herein and its pilot test results are presented. It is moderately reliable over time (intraclass correlation coefficient = 0.68, p < 0.05), has high criterion-related validity (r xy  < 0.4) and construct validity (h 2 > 0.7). High content validity evidence was ensured through a two-stage validation method, while test item discrimination coefficient values are acceptable (r pbis  > 0.1). The method is time-efficient (measuring lasts 45 min), valid, stable, and enables holistic investigation of large sample sizes.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1

Similar content being viewed by others

References

  • Avsec, S. (2012). Metoda merjenja tehnološke pismenosti učencev 9. razreda osnovne šole, Ljubljana: Univerza v Ljubljani. Retrived from: http://pefprints.pef.uni-lj.si/663

  • Bordens, K. S., & Abbott, B. B. (2011). Research design and methods: a process approach. New York: McGraw-Hill.

    Google Scholar 

  • Castillo, M. (2010). Technological literacy: Designing and testing an instrument to measure eighth-grade achievement in technology education. The American society for engineering education. Louisville, KY: Chapman & Hall/CRC.

    Google Scholar 

  • Cohen, L., Manion, L., & Morrison, K. (2007). Research methods in education. London, NewYork: Routledge Kegan Paul.

    Google Scholar 

  • Crocker, L., & Algina, J. (2008). Introduction to classical and modern test theory. Mason, Ohio: Cengage Learning.

    Google Scholar 

  • Custer, R. L., Valesey, B. G., & Burke, B. N. (2001). An assessment model for a design approach to technological problem solving. Journal for Technology Education, 12(2), 5–20.

    Google Scholar 

  • Dakers, J. R. (2006). Defining technological literacy: Towards an epistemological framework. New York: Palgrave Macmillan.

    Book  Google Scholar 

  • de Vries, M. J. (2006). Technological knowledge and artifacts: An analytical view. In J. R. Dakers (Ed.), Defining technological literacy: Towards an epistemological framework (pp. 17–30). New York: Palgrave Macmillan.

    Google Scholar 

  • DeMiranda, M. (2004). The grounding of a discipline: Cognition and instruction in technology education. International Journal of Technology and Design Education, 14, 61–77.

    Article  Google Scholar 

  • Dugger, W. E., & Gilberti, A. F. (2007). Standards for technological literacy: Content for the study of technology. Virginia: International Technology Education Association (ITEA).

    Google Scholar 

  • Eisenkraft, A. (2010). Retrospective analysis of technological literacy of K-12 students in the USA. International Journal of Technology and Design Education, 20, 277–303.

    Article  Google Scholar 

  • Frank, M. (2005). A systems approach for developing technological literacy. Journal of Technology Education, 17(1), 19–34.

    Google Scholar 

  • Gagel, W. C. (2004). Technology profile: An assessment strategy for technological literacy. The Journal of Technology Studies, 30(4), 38–44.

    Google Scholar 

  • Garmire, E., & Pearson, G. (Eds.). (2006). Tech tally: Approaches to assessing technological literacy. Washington, DC: National Academies Press.

    Google Scholar 

  • Gliner, J. A., & Morgan, G. A. (2000). Research methods in applied settings: An integrated approach to design and analysis. Mahwah, NJ: L. Erlbaum.

    Google Scholar 

  • Hayden, M. A. (1989). What is technological literacy? Bulletin of Science, Technology and Society, 9, 228–233.

    Article  Google Scholar 

  • Haynie, W. J. (2007). Effects of test taking on retention learning in technology education: A meta-analysis. Journal of Technology Education, 18(2), 24–36.

    Google Scholar 

  • Hilton, J. K. (2006). The effect of technology on student science achievement. In E. Alkhalifa (Ed.), Cognitively informed systems: Utilizing practical approaches to enrich information presentation and transfer (pp. 312–333). Hershey: Idea Group Inc.

    Chapter  Google Scholar 

  • Hodge, D., & Gillespie, D. (2007). Phrase completion scales: A better measurement approach than likert scales? Journal of Social Service Research, 33(4), 1–12.

    Article  Google Scholar 

  • Ingerman, A., & Collier-Reed, B. (2011). Technological literacy reconsidered: A model for enactment. Intenatinal Journal for Technology and Design Education, 21, 137–148.

    Article  Google Scholar 

  • International Technology and Engineering Education Association ITEEA. (2007). Standards for technological literacy: Content for the study of technology. Reston, VA: Author.

    Google Scholar 

  • Kelley, T. R. (2008). Cognitive processes of students participating in engineering. Journal of Technology Education, 19, 50–64.

    Google Scholar 

  • Kelley, T. R., & Wicklein, R. C. (2009). Examination of assessment practices for engineering design projects in secondary education (Second in a 3-part series). Journal of Industrial Teacher Education, 46(2), 6–25.

    Google Scholar 

  • Kubiszyn, T., & Borich, G. D. (2013). Educational testing and measurement: Classroom application and practise. Hoboken, NJ: Willey.

    Google Scholar 

  • Linacre, J. (2008). The expected value of a point-biserial (or similar) correlation. Rasch Measurement Transactions, 22(1), 1154–1157.

    Google Scholar 

  • Mawson, B. (2006). Factors affecting learning in technology in the early years at school. International Journal of Technology and Design Education, 17, 253–269.

    Article  Google Scholar 

  • McLaren, S. V. (2007). An international overview of assessment issues in technology education: Disentangling the influences, confusion and complexities. Design and Technology Education: An International Journal, 12(2), 2007.

    Google Scholar 

  • McMillan, J. H., & Schumacher, S. (2006). Research in education: A conceptual introduction (6th ed.). Boston: Pearson Education Inc.

  • Miller, M. D., Linn, R. L., & Gronlund, N. E. (2009). Measurement and assessment in teaching (10th ed.). New Jersey: Pearson Education ltd.

    Google Scholar 

  • Odom, L. R., & Morrow, J. R. (2006). What’s this r? A correlational approach to explaining validity, reliability and objectivity coefficients. Measurement in Physical Education and Exercise Science, 10(2), 137–145.

    Article  Google Scholar 

  • Osterlind, S. J. (1998). Constructing test items: Multiple-choice, constructed-response, performance, and other formats. Boston: Kluwer Academic Publishers.

    Google Scholar 

  • Petrina, S. (2000). The politics of technological literacy. International Journal of Technology and Design Education, 10(2), 181–206.

    Article  Google Scholar 

  • Rohaan, E. J., Taconis, R., & Jochems, W. M. G. (2010). Analysing teacher knowledge for technology education in primary schools. International Journal of Technology and Design Education,. doi:10.1007/s10798-010-9147-z.

    Google Scholar 

  • Rose, M. A. (2007). Perceptions of technological literacy among science, technology, engineering, and mathematics leaders. Journal of Technology Education, 19(1), 35–52.

    Google Scholar 

  • Rossiter, J. R. (2011). Measurement for the social sciences: The C-OAR-SE method and why it must replace psychometrics. New York: Springer.

    Book  Google Scholar 

  • Shadish, W. S., Cook, T. C., & Campbell, D. T. (2002). Experimental and quasi-experimental designs for generalized causal inference. Boston: Houghton Co.

    Google Scholar 

  • Shumway, S. L., Saunders, W., Stewardson, G., & Reeve, E. (2001). A comparison of classroom interpersonal goal structures and their effect on group problem-solving performance and student attitudes toward their learning environment. Journal of Industrial Teacher Education, 38(3), 6–24.

    Google Scholar 

  • Stiggins, R., Rubel, E., & Quellmalz, E. (1988). Measuring thinking skills in the classroom. Washington, DC: NAE, Professional Library.

    Google Scholar 

  • Stobaugh, R. (2013). Assessing critical thinking in elementary schools: meeting the common core. Larchmont, NY: Eye on Education.

    Google Scholar 

  • Suen, H. K., & McClellan, S. (2003). Item construction principles and techniques. In N. Huang (Ed.), Encyclopedia of vocational and technological education (1st ed., pp. 777–798). Taipei: ROC Ministry of Education.

    Google Scholar 

  • Taylor, J. S. (2006). Student perceptions of selected technology student association activities. Journal of Technology Education, 17(2), 56–71.

    Google Scholar 

  • Tobin, K., & Capie, W. (1981). The development and validation of a group test of logical thinking. Educational and Psychological Measurement, 41(4), 413–423.

    Article  Google Scholar 

  • Watson, G., & Glaser, E. M. (2009). Watson-glaser critical thinking apraisal manual. San Antonio: Pearson Inc.

    Google Scholar 

  • Weir, J. P. (2005). Quantifying test-retest reliability using the intraclass correlation coefficient. Journal of Strength and Conditioning Research, 19(1), 231–240.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Faculty of Education, University of Ljubljana, Ljubljana, Slovenia

    Stanislav Avsec & Janez Jamšek

Authors
  1. Stanislav Avsec
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Janez Jamšek
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Stanislav Avsec.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Avsec, S., Jamšek, J. Technological literacy for students aged 6–18: a new method for holistic measuring of knowledge, capabilities, critical thinking and decision-making. Int J Technol Des Educ 26, 43–60 (2016). https://doi.org/10.1007/s10798-015-9299-y

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10798-015-9299-y

Keywords

Search

Navigation