Skip to main content
SpringerLink
Log in

Building the BIKE: Development and Testing of the Biotechnology Instrument for Knowledge Elicitation (BIKE)

  • Published:
Research in Science Education Aims and scope Submit manuscript

Abstract

Identifying students’ conceptual scientific understanding is difficult if the appropriate tools are not available for educators. Concept inventories have become a popular tool to assess student understanding; however, traditionally, they are multiple choice tests. International science education standard documents advocate that assessments should be reform based, contain diverse question types, and should align with instructional approaches. To date, no instrument of this type targeting student conceptions in biotechnology has been developed. We report here the development, testing, and validation of a 35-item Biotechnology Instrument for Knowledge Elicitation (BIKE) that includes a mix of question types. The BIKE was designed to elicit student thinking and a variety of conceptual understandings, as opposed to testing closed-ended responses. The design phase contained nine steps including a literature search for content, student interviews, a pilot test, as well as expert review. Data from 175 students over two semesters, including 16 student interviews and six expert reviewers (professors from six different institutions), were used to validate the instrument. Cronbach’s alpha on the pre/posttest was 0.664 and 0.668, respectively, indicating the BIKE has internal consistency. Cohen’s kappa for inter-rater reliability among the 6,525 total items was 0.684 indicating substantial agreement among scorers. Item analysis demonstrated that the items were challenging, there was discrimination among the individual items, and there was alignment with research-based design principles for construct validity. This study provides a reliable and valid conceptual understanding instrument in the understudied area of biotechnology.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2

Similar content being viewed by others

References

  • Achieve Inc. (2013). The next generation science standards. Released April 2013, available from http://www.nextgenscience.org.

  • AgBio World Foundation. (2003). Biotech food myths, misconceptions and misinformation: a response to false activist claims. Retrieved August 10, 2010 from http://www.agbioworld.org/biotech-info/articles/agbio-articles/GMmyths.html.

  • Anderson, D. L., Fisher, K. M., & Norman, G. J. (2002). Development and evaluation of the conceptual inventory of natural selection. Journal of Research in Science Teaching, 39, 952–978.

    Article  Google Scholar 

  • Bowling, B. V., Acra, E. E., Wang, L., Myers, M. F., Dean, G. E., Markle, G. C., Moskalik, C. L., & Huether, C. A. (2008). Development and evaluation of a genetics literacy assessment instrument for undergraduates. Genetics, 178, 15–22.

    Article  Google Scholar 

  • Bloom, B. S. (1956). Taxonomy of educational objectives, the classification of educational goals—handbook I: cognitive domain. New York: McKay.

    Google Scholar 

  • Brown, D. E. (1992). Using examples and analogies to remediate misconceptions in physics: factors influencing conceptual change. Journal of Research in Science Teaching, 29, 17–34.

    Article  Google Scholar 

  • Caleon, I., & Subramaniam, R. (2010a). Development and application of a three-tier diagnostic test to assess secondary students’ understanding of waves. International Journal of Science Education, 32, 939–961.

    Article  Google Scholar 

  • Caleon, I., & Subramaniam, R. (2010b). Do students know what they know and what don’t know? Using a four-tier diagnostic test to assess the nature of students’ alternative conceptions. Research in Science Education, 40, 313–337.

    Article  Google Scholar 

  • Carnegie Foundation for the Advancement of Teaching. (2010). Carnegie classifications. Retrieved September 28, 2010 from http://classifications.carnegiefoundation.org.

  • Chandrasegaran, A. L., Treagust, D. F., & Mocerino, M. (2007). The development of a two-tier multiple-choice diagnostic instrument for evaluating secondary school students’ ability to describe and explain chemical reactions using multiple levels of representation. Chemistry Education Research and Practice, 8(3), 293–307.

    Article  Google Scholar 

  • Chang, C. Y., Yeh, T. K., & Barufaldi, J. P. (2010). The positive and negative effects of science concept tests on student conceptual understanding. International Journal of Science Education, 32, 265–282.

    Article  Google Scholar 

  • Cole, K., Coffey, J., & Goldman, S. (1999). Using assessments to improve equity in mathematics. Educational Leadership, 56, 56–58.

    Google Scholar 

  • Concannon, J., Siegel, M. A., Halverson, K. L., & Freyermuth, S. K. (2010). College students’ conceptions of stem cells, stem cell research, and cloning. Journal of Science Education and Technology, 19(2), 177–186.

    Google Scholar 

  • CRESST. (2001). Policy brief no. 4. Los Angeles, CA: Center for Research on Evaluation, Standards, and Student Testing (CRESST).

  • Creswell, J. W. (2009). Research design: qualitative, quantitative, and mixed methods approaches. Thousand Oaks, CA: Sage Publications, Inc.

    Google Scholar 

  • D’Avanzo, C. (2008). Biology concept inventories: overview, status, and next steps. BioScience, 58, 1079–1085.

    Article  Google Scholar 

  • Davies, A. (2003). Learning through assessment: assessment for learning in the science classroom. In J. M. Atkin & J. E. Coffey (Eds.), Everyday assessment in the science classroom (pp. 13–25). Virginia: NSTA Press.

    Google Scholar 

  • Dawson, V. M., & Schibeci, R. A. (2003). West Australian school students’ understanding of biotechnology. International Journal of Science Education, 25, 57–69.

    Article  Google Scholar 

  • Elmesky, R. (2013). Building capacity in understanding foundational biology concepts: a K-12 learning progression in genetics informed by research on children’s thinking and learning. Research in Science Education, 43, 1155–1175.

    Article  Google Scholar 

  • Gardner, G. E., & Jones, M. G. (2011). Science instructors’ perceptions of the risks of biotechnology: implications for science education. Research in Science Education, 41, 711–738.

    Article  Google Scholar 

  • Garvin-Doxas, K., Klymkowsky, M. W., & Elrod, S. (2007). Building, using, and maximizing the impact of concept inventories in the biological sciences: report on a National Science Foundation sponsored conference on the construction of concept inventories in the biological sciences. Cell Biology Education, 6, 277–282.

    Article  Google Scholar 

  • Halverson, K. L., Siegel, M. A., & Freyermuth, S. K. (2009). Lenses for framing decisions: Undergraduates’ decision making about stem cell research. International Journal of Science Education, 31(9), 1249–1268.

    Google Scholar 

  • Halverson, K. L., Freyermuth, S. K., Siegel, M. A., & Clark, C. (2010). What undergraduates misunderstand about stem cell research. International Journal of Science Education, 32(17), 2253–2272.

    Google Scholar 

  • Hankins, M. (2007). Questionnaire discrimination: (re)-introducing coefficient δ. MBC Medical Research Methodology, 7(19), 1–5.

    Google Scholar 

  • Heady, J. E. (2004). Using pretests and posttests. Teaching tips: innovations in undergraduate science instruction. Arlington, VA: NSTA Press.

    Google Scholar 

  • Hestenes, D., Wells, M., & Swackhamer, G. (1992). Force concept inventory. The Physics Teacher, 30, 141–166.

    Article  Google Scholar 

  • Hmelo-Silver, C. E., & Duncan, R. G. (2009). Learning progressions. Journal of Research in Science Teaching, 46, 606–609.

    Article  Google Scholar 

  • Klymkowsky, M.W., & Garvin-Doxas, K. (2007). Bioliteracy: building the biology concept inventory. http://bioliteracy.net.

  • Landis, J. R., & Koch, G. G. (1977). The measurement of observer agreement for categorical data. Biometrics, 33, 59–174.

    Google Scholar 

  • Light, R. J. (2001). Making the most of college: students speak their minds. Cambridge, MA: Harvard University Press.

    Google Scholar 

  • Liu, O. L., Lee, H., Hofstetter, C., & Linn, M. C. (2008). Assessing knowledge integration in science: construct, measures, and evidence. Educational Assessment, 13, 33–55.

    Article  Google Scholar 

  • Lord, T. R., French, D. P., & Crow, L. W. (2009). College science teachers guide to assessment. Virginia: NSTA Press.

    Google Scholar 

  • Lyons, R. E., McIntosh, M., & Kysilka, M. L. (2003). Teaching college in the age of accountability. Boston, MA: Allyn and Bacon.

    Google Scholar 

  • Mascazine, J.R., Titterington, L., Khalaf, A.K. (1998). Cloning: what do they know? A report on the general knowledge of a sample of Midwestern citizens. Paper presented at the Annual Meeting of the National Association for Research in Science Teaching, San Diego, CA.

  • Mulford, D. R., & Robinson, W. R. (2002). An inventory for alternate conceptions among first semester general chemistry students. Journal of Chemistry Education, 79, 739–744.

    Article  Google Scholar 

  • National Research Council. (1996). National Science Education Standards. Washington, DC: National Academy Press.

    Google Scholar 

  • National Research Council. (2001). Classroom assessment and the National Science Education Standards. Washington, DC: National Academy Press.

    Google Scholar 

  • National Research Council. (2002). Animal biotechnology: science-based concerns. Washington, DC: National Academy Press.

    Google Scholar 

  • National Research Council. (2004). Safety of genetically engineered foods: approaches to assessing unintended health effects. Washington, DC: National Academy Press.

    Google Scholar 

  • National Research Council. (2008). Global challenges and directions for agricultural biotechnology. Washington, DC: National Academy Press.

    Google Scholar 

  • National Research Council. (2009a). A new biology for the 21st century. Washington, DC: National Academy Press.

    Google Scholar 

  • National Research Council. (2009b). Engineering in K-12 education. Washington, DC: National Academy Press.

    Google Scholar 

  • National Research Council. (2010). Standards for K-12 engineering education? Washington, DC: National Academy Press.

    Google Scholar 

  • National Research Council. (2011). Promising practices in undergraduate science, technology, engineering, and mathematics education: summary of two workshops. Washington, DC: National Academy Press.

    Google Scholar 

  • National Research Council. (2012). A framework for K-12 science education: practices, cross-cutting concepts, and core ideas. Washington, DC: National Academy Press.

    Google Scholar 

  • National Science Foundation. (1996). Shaping the future: new expectations for undergraduate education in science, mathematics, engineering, and technology (NSF Report No. 96–139). Washington DC: National Science Foundation.

  • Novak, J. (1991). Clarify with concept maps. The Science Teacher, 58, 44–49.

    Google Scholar 

  • Nunnally, J. (1978). Psychometric theory. New York: McGraw-Hill.

    Google Scholar 

  • Odom, A. L., & Barrow, L. H. (1995). Development and application of a two-tier diagnostic test measuring college biology students’ understanding of diffusion and osmosis after a course of instruction. Journal of Research in Science Teaching, 32, 45–61.

    Article  Google Scholar 

  • Oosterhof, A. (1996). Developing and using classroom assessment. Englewood Cliffs: Merrill/Prentice Hall.

    Google Scholar 

  • Othman, J., Treagust, D. F., & Chandrasegaran, A. L. (2008). An investigation into the relationship between students’ conceptions of the particulate nature of matter and their understanding of chemical bonding. International Journal of Science Education, 30(11), 1531–1550.

    Article  Google Scholar 

  • Plummer, J. D., & Krajcik, J. (2010). Building a learning progression for celestial motion: elementary levels from an earth-based perspective. Journal of Research in Science Teaching, 47, 768–787.

    Article  Google Scholar 

  • Rebello, C. M., Siegel, M. A., Freyermuth S. K., Witzig, S. B., & Izci, K. (2012). Development of embedded assessments for learning in biotechnology: results and design process for dissemination. Biochemistry and Molecular Biology Education, 40(2), 82–88.

    Google Scholar 

  • Richardson, J. (2005). Concept inventories: tools for uncovering STEM students’ misconceptions. In: Invention and impact: building excellence in undergraduate science, technology, engineering and mathematics (STEM) education (pp. 19–25). Washington (DC): American Association for the Advancement of Science.

  • Sadler, P. M. (1998). Psychometric models of student conceptions in science: reconciling qualitative studies and distracter-driven assessment instruments. Journal of Research in Science Teaching, 35, 265–298.

    Article  Google Scholar 

  • Shaw, K. R., Horne, K. V., Zhang, H., & Boughman, J. (2008). Essay contest reveals misconceptions of high school students in genetics contest. Genetics, 178, 1157–1168.

    Article  Google Scholar 

  • Siegel, M. A. (2007). Striving for equitable classroom assessments for linguistic minorities: Strategies for and effects of revising life science items. Journal of Research in Science Teaching, 44(6), 864–881.

    Google Scholar 

  • Siegel, M. A., Wissehr, C., & Halverson, K. L. (2008). Sounds like success: a framework for equitable assessment. The Science Teacher, 75(3), 43–46.

    Google Scholar 

  • Smith, C. L., Wiser, M., Anderson, C. W., & Krajcik, J. (2006). Implications of research on children’s learning for standards and assessment: a proposed learning progression for matter and the atomic molecular theory. Measurement: Interdisciplinary Research and Perspectives, 14, 1–98.

    Google Scholar 

  • Smith, M. K., Wood, W. B., & Knight, J. K. (2008). The genetics concept assessment: a new concept inventory for gauging student understanding of genetics. CBE Life Sciences Education, 7, 422–430.

    Article  Google Scholar 

  • SPSS. (2008). SPSS for Windows (release 16.0) [computer software]. Chicago: SPSS.

    Google Scholar 

  • Tan, K. C. D., Goh, N. K., Chia, L. S., & Treagust, D. F. (2002). Development and application of a two-tier multiple-choice diagnostic instrument to assess high school students’ understanding of inorganic chemistry qualitative analysis. Journal of Research in Science Teaching, 39, 283–301.

    Article  Google Scholar 

  • Thieman, W. J., & Palladino, M. A. (2004). Introduction to biotechnology. San Fransisco, CA: Pearson Education Inc.

    Google Scholar 

  • Treagust, D. F. (1988). Development and use of diagnostic tests to evaluate students’ misconceptions in science. International Journal of Science Education, 10, 159–169.

    Article  Google Scholar 

  • Treagust, D.F., Crowley, J., Mocerino, M., Chandrasegaran, A.L. (2011). Persistent student difficulties in understanding the particulate nature of matter. Paper presented at the National Association for Research in Science Teaching annual meeting, Orlando, FL.

  • Tsui, C., & Treagust, D. (2010). Evaluating secondary students’ scientific reasoning in genetics using a two-tier diagnostic instrument. International Journal of Science Education, 32, 1073–1098.

    Article  Google Scholar 

  • Wandersee, J. H., Mintzes, J. J., & Novak, J. D. (1994). Research on alternative conceptions in science. In D. Gabel (Ed.), Handbook of research on science teaching and learning (pp. 177–210). New York: Simon & Schuster Macmillan.

    Google Scholar 

  • White, B. Y., & Frederiksen, J. R. (1998). Inquiry, modeling, and metacognition: making science accessible to all students. Cognition and Instruction, 16, 3–118.

    Article  Google Scholar 

  • Witzig, S. B., Freyermuth, S. K., Siegel, M. A., Izci, K., & Pires, J. C. (2013). Is DNA alive? A study of conceptual change through targeted instruction. Research in Science Education, 43(4), 1361–1375.

    Google Scholar 

  • Wood, D., Brunner, J. S., & Ross, G. (1976). The role of tutoring and problem solving. Journal of Child Psychology and Psychiatry, 17, 89–100.

    Article  Google Scholar 

Download references

Acknowledgments

This material is based on work supported by the National Science Foundation (NSF) under Grant No. 0837021. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the NSF.

Author information

Authors and Affiliations

  1. Department of STEM Education & Teacher Development, University of Massachusetts Dartmouth, 398E Liberal Arts Building, 285 Old Westport Road, Dartmouth, MA, 02747, USA

    Stephen B. Witzig

  2. Office of Educational Innovation and Evaluation, Kansas State University, Manhattan, KS, USA

    Carina M. Rebello

  3. MU Science Education Center, University of Missouri (MU), Columbia, MO, USA

    Marcelle A. Siegel

  4. Department of Biochemistry, University of Missouri (MU), Columbia, MO, USA

    Marcelle A. Siegel, Sharyn K. Freyermuth & Bruce McClure

  5. College of Education, Department of Science Education, Van Yuzuncuyıl University, Van, Turkey

    Kemal Izci

Authors
  1. Stephen B. Witzig
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Carina M. Rebello
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Marcelle A. Siegel
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Sharyn K. Freyermuth
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Kemal Izci
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Bruce McClure
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Stephen B. Witzig.

Electronic Supplementary Material

Below is the link to the electronic supplementary material.

Fig. S1

(DOC 40 kb)

Fig. S2

(DOC 61 kb)

Rights and permissions

Reprints and permissions

About this article

Cite this article

Witzig, S.B., Rebello, C.M., Siegel, M.A. et al. Building the BIKE: Development and Testing of the Biotechnology Instrument for Knowledge Elicitation (BIKE). Res Sci Educ 44, 675–698 (2014). https://doi.org/10.1007/s11165-014-9398-x

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11165-014-9398-x

Keywords

Search

Navigation