Design of Online Professional Development in Science Content and Pedagogy: A Pilot Study in Florida

Abstract

The Exploring Florida Science project goals were: (1) increasing content knowledge of secondary science teachers, specifically in topics that are important to the future of the state, and (2) providing secondary science students with digital media for use in project-based learning. A team of instructional designers, educators, scientists, web designers and teacher educators designed an online professional development environment. Needs analysis included identification of the high incidence topics on the state science achievement test and topics of socio-scientific importance in the state. Development followed tenets of design-based research, and was guided by E-Learning for Educators standards and evaluated for content, pedagogy and usability using rubrics based on established guidelines. This paper details design, development, and evaluation frameworks, and summarizes pilot testing outcomes.

Appendix

Science Pedagogy Evaluation Rubric

Science Pedagogy

This rubric is used by science experts—science teacher educators or doctoral level students in science education—to review the pedagogical aspects of the module materials. The rubric is used after the early versions of the teaching materials (teaching strategies, FCAT items, learning assessments, standards and benchmarks) are developed. The primary question addressed by this rubric is, “To what degree are the pedagogical practices of the module consistent with current educational theory and practice?” Please rate each module according to the following criteria and recommend changes and additions to the module.

II. Science Pedagogy Rubric To what extent does this module meet each listed criterion?	Great extent 5	4	Some-what 3	2	Very little 1	Strengths noted	Limitations noted/suggested additions or revisions
(A) Clarity of module goals and learning objectives The module’s goals and learning objectives are explicit and clearly communicated	O	O	O	O	O
(B) Applicability of the objectives and materials The objectives and materials presented in the module are appropriate for the science classroom for the grade level and standards indicated, with no examples outside of the topic or grade range	O	O	O	O	O
(C) Alignment of content with benchmarks The content aligns completely with the designated benchmarks, with no gaps or overlap	O	O	O	O	O
(D) Alignment of module components The module’s goals, objectives, activities, and assessments are tightly aligned	O	O	O	O	O
(E) Consistency with current theory and practice for inquiry teaching The strategies used in module lessons strongly foster student-centered inquiry-based science teaching	O	O	O	O	O
(F) Adequacy of instructions and explanations The instructions and explanations for teachers and students are complete, clear and at the appropriate level for the intended user	O	O	O	O	O
(G) Ability to motivate the audience The module materials use media in ways that engage users and enhance learning	O	O	O	O	O

Science Content Evaluation Rubric

Science Content

This rubric is used by science content experts—scientists with background in the topic—to review the content of module materials. The rubric is used after the early drafts of the content materials (video, science resources, and lessons) are developed. The primary question addressed by this rubric is, “To what degree is the science content of the module materials – including background material, datasets, audio/video media, lesson descriptions, and sample assessment items – consistent with current scientific thinking?” Please rate each module according to the following criteria and recommend changes and additions to the module.

I. Science Content Rubric II. To what extent does this module meet each listed criterion?	Great extent 5	4	Some-what 3	2	Very little 1	Strengths noted	Limitations noted/suggested additions or revisions
(A) Accuracy of the content Content is accurate with no errors of scientific fact or interpretation	O	O	O	O	O
(B) Currency of scientific content Content reflects the current state of the science, with no instances in which significant updating is needed	O	O	O	O	O
(C) Completeness for the grade level For the grade level indicated, the content is presented at the appropriate level with no examples significantly above or below grade level	O	O	O	O	O
(D) Thoroughness for benchmarks All of the science benchmarks designated for the module are addressed with appropriate depth and balance	O	O	O	O	O
(E) Appropriateness of the science skills The science skills required in the module address Strand H, Processes of Science, at the grade level indicated and are appropriate to the science discipline in which they are applied	O	O	O	O	O

Delivery Technology Evaluation Rubric

Technology for Delivery

This rubric is used by expert distance educators—graduate level instructional designers and professionals in distance education programs—to review the delivery methods of the modules. The rubric is used after early versions of the online materials are developed. The primary question addressed by this rubric is, “To what degree is the module delivery system consistent with current effective practice in online professional development?” Please rate each module according to the following criteria and recommend changes and additions to the module.

III. Technology Rubric To what extent does this module meet each listed criterion?	Great extent 5	4	Some-what 3	2	Very little 1	Strengths noted	Limitations noted/suggested additions or revisions
(A) Usability of the media by the intended audience Florida science teachers can successfully use the module materials with school technology and typical skills	O	O	O	O	O
(B) Clarity of visual design The module’s aesthetic design presents and communicates information clearly throughout the module	O	O	O	O	O
(C) Efficiency of navigation The module is well-organized, visually and functionally consistent, and easy for teachers to navigate	O	O	O	O	O
(D) Professionalism of media The module’s media are attractive, functional, and appropriate for teachers	O	O	O	O	O
(E) Accessibility of materials for all audiences All module components are free from bias and accessible to teachers of various physical abilities (or have appropriate accommodations)	O	O	O	O	O
(F) Adequacy of support provided for users Adequate technical support is provided to enable users to work independently	O	O	O	O	O