Skip to main content
SpringerLink
Log in

Exploring Students’ Scientific Competency Performance on PISA Paper-Based Assessment and Computer-Based Assessment

  • Chapter
  • First Online:
Learning from Animations in Science Education

Part of the book series: Innovations in Science Education and Technology ((ISET,volume 25))

  • 699 Accesses

Abstract

With the use of PISA test items, this chapter explores how similar background students’ scientific competency performance differed on paper-based and computer-based assessments. A nationally representative sample of 15-year-old Taiwan students (N = 3288) from the PISA 2015 Field Trial test were selected to participate in the study. In addition to the PISA standard contextualized units, which consisted of static materials including text, graphs, and tables, interactive units including animations and simulations were used in the competency assessment. Three groups of students answered static trend items with a paper-based assessment, static trend items with a computer-based assessment, and new interactive animation and simulation items with a computer-based assessment. The results of their performance were analysed and are discussed as well as the role of information and communication technology familiarity in scientific competency. Finally, the implications of animation and simulation in the learning and assessment of scientific competency are discussed.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 139.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 179.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 179.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  • Ainsworth, S. (1999). The functions of multiple representations. Computers & Education, 33(2), 131–152.

    Article  Google Scholar 

  • Ainsworth, S. (2006). DeFT: A conceptual framework for considering learning with multiple representations. Learning and Instruction, 16(3), 183–198.

    Article  Google Scholar 

  • Al-Balush, S. M., & Al-Hajrib, S. H. (2014). Associating animations with concrete models to enhance students’ comprehension of different visual representations in organic chemistry. Chemistry Education Research and Practice, 15, 47–58.

    Article  Google Scholar 

  • Areepattamannil, S., & Santos, I. M. (2019). Adolescent students’ perceived information and communication technology (ICT) competence and autonomy: Examining links to dispositions toward science in 42 countries. Computers in Human Behavior, 98, 50–58.

    Article  Google Scholar 

  • Backes, B., & Cowan, J. (2019). Is the pen mightier than the keyboard? The effect of online testing on measured student achievement. Economics of Education Review, 68, 89–103.

    Article  Google Scholar 

  • Bandura, A. (1986). Social foundations of thought and action: A social cognitive theory. Englewood Cliffs, NJ: Prentice-Hall.

    Google Scholar 

  • Barak, M., & Hussein-Farraj, R. (2013). Integrating model-based learning and animations for enhancing students’ understanding of proteins structure and function. Research in Science Education, 43, 619–636.

    Article  Google Scholar 

  • Blanco-Lopez, A., Espana-Ramos, E., Gonzalez-Garcia, F. J., & Franco-Mariscal, A. J. (2015). Key aspects of scientific competence for citizenship: A Delphi study of the expert community in Spain. Journal of Research in Science Teaching, 52(2), 164–198.

    Article  Google Scholar 

  • Bong, M., & Skaalvik, E. M. (2003). Academic self-concept and self-efficacy: How different are they really? Educational Psychology Review, 15(1), 1–40.

    Article  Google Scholar 

  • Chabalengula, V. M., Sanders, M., & Mumba, F. (2012). Diagnosing students’ understanding of energy and its related concepts in biological context. International Journal of Science and Mathematics Education, 10(2), 241–266. https://doi.org/10.1007/s10763-011-9291-2

    Article  Google Scholar 

  • Chang, H. Y., & Tzeng, S. F. (2018). Investigating Taiwanese students’ visualization competence of matter at the particulate level. International Journal of Science and Mathematics Education, 16(7), 1207–1266.

    Article  Google Scholar 

  • Cheema, J. R., & Hang, B. (2013). Quantity and quality of computer use and academic achievement: Evidence from a large-scale international test program. International Journal of Education and Development using Information and Communication Technology, 9(2), 95–106.

    Google Scholar 

  • Chi, S., Wang, Z., Liu, X., & Zhu, L. (2017). Associations among attitudes, perceived difficulty of learning science, gender, parents’ occupation and students’ scientific competencies. International Journal of Science Education, 39(16), 2171–2188.

    Article  Google Scholar 

  • Chiu, J. L., DeJaegher, C. J., & Chao, J. (2015). The effects of augmented virtual science laboratories on middle school students’ understanding of gas properties. Computers & Education, 85, 59–73.

    Article  Google Scholar 

  • Chiu, M.-H., & Duit, R. (2011). Globalization: Science education from an international perspective. Journal of Research in Science Teaching, 48(6), 553–566.

    Article  Google Scholar 

  • Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Academic.

    Google Scholar 

  • Csapo, B., Ainley, J., Bennett, R. E., Latour, T., & Law, N. (2012). Technological issues for computer-based assessments. In P. Griffin, B. McGraw, & E. Care (Eds.), Assessment and teaching of 21st century skills (pp. 143–230). New York: Springer.

    Chapter  Google Scholar 

  • de Jong, T., Linn, M. C., & Zacharias, Z. C. (2013). Physical and virtual laboratories in science and engineering education. Science, 340(6130), 305–308.

    Article  Google Scholar 

  • DeBoer, G. E. (2011). The globalization of science education. Journal of Research in Science Teaching, 48(6), 567–591.

    Article  Google Scholar 

  • Dickhäuser, O., & Stiensmeier-Pelster, J. (2003). Gender differences in choice of computer courses: Applying an expectancy-value model. Social Psychology of Education, 6, 173–189.

    Article  Google Scholar 

  • Dohn, N. B. (2013). Upper secondary students’ situational interest: A case study of the role of a zoo visit in a biology class. International Journal of Science Education, 35(16), 2732–2751.

    Article  Google Scholar 

  • Dolin, J. (2007). Science education standards and science assessment in Denmark. In D. Waddington, P. Nentwig, & S. Schanze (Eds.), Making it comparable: Standards in science education (pp. 71–82). Munster: Waxmann.

    Google Scholar 

  • European Council. (2006). Key competences for lifelong learning – European reference framework. Luxembourg: Author. Available from http://www.britishcouncil.org/sites/default/files/youth-in-action-keycomp-en.pdf

  • Fallon, G. (2019). Using simulations to teach young students science concepts: An experiential learning theoretical analysis. Computers & Education, 135, 138–159.

    Article  Google Scholar 

  • Fensham, P. (2009). Real world contexts in PISA science: Implications for context-based science education. Journal or Research in Science Teaching, 46(8), 884–896.

    Article  Google Scholar 

  • Fraillon, J., Ainley, J., Schulz, W., Friedman, T., & Gebhardt, E. (2014). Preparing for life in a digital age. Singapore: Springer.

    Book  Google Scholar 

  • Freeman, B., Marginson, S., & Tytler, R. (Eds.). (2014). The age of STEM: Educational policy and practice across the world in science, technology, engineering and mathematics. New York: Routledge.

    Google Scholar 

  • Gale, J., Wind, S., Koval, J., Dagosta, J., Ryan, M., & Usselman, M. (2016). Simulation-based performance assessment: An innovative approach to exploring understanding of physical science concepts. International of Science Education, 38(14), 2284–2302.

    Article  Google Scholar 

  • Goldhammer, F., Gniewosz, G., & Zylka, J. (2016). ICT engagement in learning environments. In S. Kuger, E. Klieme, N. Jude, & D. Kaplan (Eds.), Assessing contexts of learning: An international perspective (pp. 331–351). Singapore: Springer.

    Chapter  Google Scholar 

  • Gordon, I. J. (1968). Test manual for the how I see myself scale. Gainesville: Florida Educational Research and Development Council.

    Google Scholar 

  • Gumus, S., & Atalmis, E. H. (2011). Exploring the relationship between purpose of computer usage and reading skills of Turkish students: Evidence from PISA 2006. TOJET – Turkish Online Journal of Educational Technology, 10(3), 129–140.

    Google Scholar 

  • Guzmán, A., Oliveros, D., & Mendoza, M. (2017). Scientific competencies: A mechanism to favour the inclusion of working market professionals. Journal of Baltic Science Education, 16(2), 175–187.

    Article  Google Scholar 

  • Hagger, M. S., Sultan, S., Hardcastle, S. J., & Chatzisarantis, N. L. D. (2015). Perceived autonomy support and autonomous motivation toward mathematics activities in educational and out-of-school contexts is related to mathematics homework behavior and attainment. Contemporary Educational Psychology, 41, 111–123.

    Article  Google Scholar 

  • Hidi, S., & Renninger, K. A. (2006). The four-phase model of interest development. Educational Psychologist, 41(2), 111–127.

    Article  Google Scholar 

  • Hong, Z.-R., & Lin, H.-s. (2011). An investigation of students’ personality traits and attitudes toward science. International Journal of Science Education, 33(7), 1001–1028.

    Article  Google Scholar 

  • Hu, X., Gong, Y., Lai, C., & Leung, F. (2018). The relationship between ICT and student literacy in mathematics, reading, and science across 44 countries: A multilevel analysis. Computers & Education, 125, 1–13.

    Article  Google Scholar 

  • Jaakkola, T., Nurmi, S., & Veermans, K. (2011). A comparison of students’ conceptual understanding of electric circuits in simulation only and simulation-laboratory contexts. Journal of Research in Science Teaching, 48(1), 71–93.

    Article  Google Scholar 

  • Jack, B., Lin, H.-s., & Yore, L. D. (2014). The synergistic effect of affective factors on student learning outcomes. Journal of Research in Science Teaching, 51(8), 1084–1101.

    Article  Google Scholar 

  • Krathwohl, D. (2002). A revision of Bloom’s taxonomy: An overview. Theory Into Practice, 41(4), 212–218.

    Article  Google Scholar 

  • Kuo, C.-Y., & Wu, H.-K. (2013). Toward an integrated model for designing assessment systems: An analysis of the current status of computer-based assessments in science. Computers & Education, 68, 388–403.

    Article  Google Scholar 

  • Li, N., & Black, J. B. (2016). Inter-level scaffolding and sequences of representational activities in teaching a chemical system with graphical simulations. Journal of Science and Technology, 25, 715–730.

    Google Scholar 

  • Lin, H.-s., Hong, Z., & Chen, Y. (2013). Exploring the development of college students’ situational interest in learning science. International Journal of Science Education, 35(13), 2152–2173.

    Article  Google Scholar 

  • Lin, H.-s., Hong, Z.-R., & Huang, T. (2012). The role of emotional factors in building public scientific literacy and engagement with science. International Journal of Science Education, 34(1), 25–42.

    Article  Google Scholar 

  • Lin, H.-s., Lawrenz, F., Lin, S., & Hong, Z.-R. (2012). Relationships among affective factors and preferred engagement in science related activities. Public Understanding of Science, 22(8), 941–954.

    Article  Google Scholar 

  • Lin, H.-s., Shiau, B., & Lawrenz, F. (1996). The effectiveness of teaching science with pictorial analogies. Research in Science Education, 26(4), 495–511.

    Article  Google Scholar 

  • Liu, W. C., Wang, J. C. K., & Ryan, R. M. (2016). Building autonomous learners: Perspectives from research and practice using self-determination theory. Singapore: Springer.

    Book  Google Scholar 

  • Ministry of Education. (2015). Curriculum outlines of “nature science and living technology” for K–12. Taipei: Author.

    Google Scholar 

  • Moore, D. S., Notz, W. I., & Flinger, M. A. (2013). The basic practice of statistics (6th ed.). New York: W. H. Freeman.

    Google Scholar 

  • Murphy, S., MacDonald, A., Danaia, L., & Wang, C. (2018). An analysis of Australian STEM education strategies. Policy Futures in Education, 17(2), 122–139.

    Article  Google Scholar 

  • National Research Council. (2000). Inquiry and the National Science Education Standards: A guide for teaching and learning. Washington, DC: National Academies Press.

    Google Scholar 

  • National Research Council. (2012). A framework for K–12 science education: Practices, crosscutting concepts, and core ideas. Washington, DC: National Academies Press.

    Google Scholar 

  • Naumann, J. (2015). A model of online reading engagement: Linking engagement, navigation, and performance in digital reading. Computers in Human Behavior, 53, 263–277.

    Article  Google Scholar 

  • Nieswandt, M. (2007). Student affect and conceptual understanding in learning chemistry. Journal of Research in Science Teaching, 44(7), 908–937.

    Article  Google Scholar 

  • Nikou, S. A., & Economides, A. A. (2016). The impact of paper-based, computer-based and mobile-based self-assessment on students’ science motivation and achievement. Computers in Human Behavior, 55(Part B), 1241–1248.

    Article  Google Scholar 

  • OECD. (2015). Students, computers and learning: Making the connection. Paris: OECD Publishing.

    Book  Google Scholar 

  • OECD. (2016). PISA 2015 results (Vol. I): Excellence and equity in education. Paris: PISA, OECD Publishing. https://doi.org/10.1787/9789264266490-en

    Book  Google Scholar 

  • OECD. (2017). PISA 2015 assessment and analytical framework: Science, reading, mathematics, financial literacy and collaborative problem solving. Paris: OECD Publishing.

    Book  Google Scholar 

  • Olympiou, G., Zacharias, Z. C., & de Jong, T. (2013). Making the invisible visible: Enhancing students’ conceptual understanding by introducing representations of abstract objects in a simulation. Instructional Science, 41, 575–596.

    Article  Google Scholar 

  • Palmer, D. H. (2004). Situational interest and the attitudes towards science of primary teacher education students. International Journal of Science Education, 26(7), 895–908.

    Article  Google Scholar 

  • Petko, D., Cantieni, A., & Prasse, D. (2017). Perceived quality of educational technology matters: A secondary analysis of students’ ICT use, ICT-related attitudes, and PISA 2012 test scores. Journal of Educational Computing Research, 54(8), 1070–1091.

    Article  Google Scholar 

  • Plass, J. I., Milne, C., Homer, B. D., Schwartz, R. N., Hayward, E. O., Jordan, T., et al. (2012). Investigating the effectiveness of computer simulations for chemistry learning. Journal of Research in Science Teaching, 49(3), 394–419.

    Article  Google Scholar 

  • Renninger, K. A. (2000). Individual interest and its implications for understanding intrinsic motivation. In J. M. Harackiewicz & C. Sansone (Eds.), Intrinsic and extrinsic motivation: The search for optimal motivation and performance (pp. 373–404). San Diego: Academic.

    Chapter  Google Scholar 

  • Rutten, N., van Joolingen, W. R., & van der Veen, J. T. (2012). The learning effects of computer simulations in science education. Computers & Education, 58(1), 136–153.

    Article  Google Scholar 

  • Ryan, R. M., & Deci, E. L. (2000). Self-determination theory and the facilitation of intrinsic motivation, social development, and well-being. American Psychologist, 55, 68–78.

    Article  Google Scholar 

  • Ryan, R. M., & Deci, E. L. (2017). Self-determination theory: Basic psychological needs in motivation, development, and wellness. New York: Guilford.

    Book  Google Scholar 

  • Ryoo, K., & Bedell, K. (2017). Supporting linguistically diverse students’ science learning with dynamic visualization through discourse-rich practices. Journal of Research in Science Teaching, 56(3), 270–301.

    Article  Google Scholar 

  • Schumacker, R. E., & Lomax, R. G. (2010). A beginner’s guide to structural equation modeling (3rd ed.). New York: Routledge.

    Google Scholar 

  • Senkbeil, M. (2018). Development and validation of the ICT motivation scale for young adolescents. Results of the international school assessment study ICILS 2013 in Germany. Learning and Individual Differences, 67, 167–176.

    Article  Google Scholar 

  • Singh, K., Allen, K. R., Scheckler, R., & Darlington, L. (2007). Women in computer-related majors: A critical synthesis of research and theory from 1994 to 2005. Review of Educational Research, 77(4), 500–533.

    Article  Google Scholar 

  • Skryabin, M., Zhang, J., Liu, L., & Zhang, D. (2015). How the ICT development level and usage influence student achievement in reading, mathematics, and science? Computers & Education, 85, 49–58.

    Article  Google Scholar 

  • Smith, K. C., & Villarreal, S. (2015). Using animations in identifying general chemistry students’ misconceptions and evaluating their knowledge transfer relating to particle position in physical changes. Chemistry Education Research and Practice, 16, 273–282.

    Article  Google Scholar 

  • Taasoobshirazi, G., & Sinatra, G. M. (2011). A structural equation model of conceptual change in physics. Journal of Research in Science Teaching, 48(8), 901–918.

    Article  Google Scholar 

  • Tamim, R. M., Bernard, R. M., Borokhovski, E., Abrami, P. C., & Schmid, R. F. (2011). What forty years of research says about the impact of technology on learning: A second-order meta-analysis and validation study. Review of Educational Research, 81(1), 4–28.

    Article  Google Scholar 

  • United Nations Environment Programme. (2012). 21 issues for the 21st century: Result of the UNEP foresight process on emerging environmental issues. Nairobi: Author.

    Google Scholar 

  • Unsworth, L., & Chan, E. (2009). Bridging multimodal literacies and national assessment programs in literacy. Australian Journal of Language and Literacy, 32(3), 245–257.

    Google Scholar 

  • Vekiri, I. (2010). Socioeconomic differences in elementary students: ICT beliefs and out-of-school experiences. Computers & Education, 54, 941–950.

    Article  Google Scholar 

  • Velasco, M. S. (2014). Do higher education institutions make a difference in competence development? A model of competence production at university. Higher Education, 68(4), 503–523.

    Article  Google Scholar 

  • Wang, T., & Tseng, Y. (2018). The comparative effectiveness of physical, virtual, and virtual-physical manipulatives on third-grade students’ science achievement and conceptual understanding of evaporation and condensation. International Journal of Science and Mathematics Education, 16(2), 203–219.

    Article  Google Scholar 

  • Yaseen, Z. (2018). Using student-generated animations: The challenge of dynamic chemical models in states of matter and the invisibility of the particles. Chemistry Education Research and Practice, 19, 1166–1185.

    Article  Google Scholar 

  • Zylka, J., Christoph, G., Kroehne, U., Hartig, J., & Goldhammer, F. (2015). Moving beyond cognitive elements of ICT literacy: First evidence on the structure of ICT engagement. Computers in Human Behaviour, 53, 149–160.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. National Sun Yat-sen University (NSYSU), Kaohsiung City, Taiwan

    Ya-Chun Chen & Zuway-R Hong

  2. Australian Catholic University, Brisbane, QLD, Australia

    Huann-shyang Lin

Authors
  1. Ya-Chun Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Zuway-R Hong
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Huann-shyang Lin
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Huann-shyang Lin .

Editor information

Editors and Affiliations

  1. North Sydney Campus, Australian Catholic University, North Sydney, NSW, Australia

    Len Unsworth

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Chen, YC., Hong, ZR., Lin, Hs. (2020). Exploring Students’ Scientific Competency Performance on PISA Paper-Based Assessment and Computer-Based Assessment. In: Unsworth, L. (eds) Learning from Animations in Science Education. Innovations in Science Education and Technology, vol 25. Springer, Cham. https://doi.org/10.1007/978-3-030-56047-8_12

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-56047-8_12

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-56046-1

  • Online ISBN: 978-3-030-56047-8

  • eBook Packages: EducationEducation (R0)

Publish with us

Policies and ethics

Search

Navigation