The InTeGrate Materials Development Rubric: A Framework and Process for Developing Curricular Materials that Meet Ambitious Goals

  • David SteerEmail author
  • Ellen R. Iverson
  • Anne E. Egger
  • Kim A. Kastens
  • Cathryn A. Manduca
  • David McConnell
Part of the AESS Interdisciplinary Environmental Studies and Sciences Series book series (AESS)


We designed and tested a curriculum development and auditing methodology for the Interdisciplinary Teaching about Earth for a Sustainable Future (InTeGrate) project. That process was driven and facilitated by a written rubric for curriculum development. Materials developers participated in workshops to prepare them to write and revise their materials in accordance with the rubric and were guided by an assessment consultant. Other assessment team members independently audited (reviewed) the materials before they could be tested with students. Curriculum developers encountered the most difficulty meeting criteria related to metacognition, grading rubrics, writing learning outcomes and objectives, and linking and aligning materials across the curriculum. Changes to the professional development program improved teams’ abilities to meet those standards. We found the development rubric and process to be an effective methodology for developing materials addressing grand challenges facing society.


Curriculum development Rubric Professional development InTeGrate Backward design 



This work is supported by the National Science Foundation (NSF) collaboration between the Directorates for Education and Human Resources (EHR) and Geosciences (GEO) under grant DUE-1125331.

Disclaimer: Any opinions, findings, conclusions, or recommendations expressed in this paper are those of the author(s) and do not necessarily reflect the views of the National Science Foundation. The authors thank the InTeGrate module development teams and the InTeGrate Assessment Team members for their hard work and patience in pioneering a new system of peer-supported materials development.


  1. American Association for the Advancement of Science (2009) Benchmarks for science literacy. Accessed 12 Feb 2014
  2. Anderson L, Krathwohl D (eds) (2001) A taxonomy for learning, teaching and assessing: a revision of bloom’s taxonomy of educational objectives. Longman, New YorkGoogle Scholar
  3. Angelo T, Cross P (1993) Classroom assessment techniques: a handbook for college teachers. Jossey-Bass, San FranciscoGoogle Scholar
  4. Assaraf O, Orion N (2005) Development of systems thinking skills in the context of earth system education. J Res Sci Teach 42(5):518–560CrossRefGoogle Scholar
  5. Ault C, Dodick J (2010) Tracking the footprints puzzle: the problematic persistence of science-as-process in teaching the nature and culture of science. Sci Educ 94(6):1092–1122CrossRefGoogle Scholar
  6. Bhattacharyya P, Branlund J, Joseph L (2014) Humans’ dependence on earth’s mineral resources. Accessed 25 June 2018
  7. Bell R (2004) Perusing Pandora’s Box: exploring the what, when and how of nature of science instruction. In: Flick L, Lederman N (eds) Scientific inquiry and the nature of science: implications for teaching, learning, and teacher education. Kluwer Academic Publishers, Dordrecht, pp 427–446Google Scholar
  8. Biggs J (1996) Enhanced teaching through constructive alignment. High Educ 32:347–364CrossRefGoogle Scholar
  9. Biggs J (2003) Aligning teaching and assessing to course objectives. In: Teaching and learning in higher education: new trends and innovations. University of Aveiro, Aveiro, pp 13–17Google Scholar
  10. Black P, Wiliam D (1998) Assessment and classroom learning. Assess Educ 5(1):7–74CrossRefGoogle Scholar
  11. (2012) Exemplary course program [online]. Available at Accessed 18 May 2018
  12. Boyle B, Charles M (2014) Formative assessment for teaching and learning. Sage, LondonCrossRefGoogle Scholar
  13. Bransford J (2000) How people learn: brain, mind, experience, and school. National Academy Press, Washington, DCGoogle Scholar
  14. Bransford J, Schwartz D (1999) Rethinking transfer: a simple proposal with multiple implications. In: Iran-Nejad A, Pearson P (eds) Review of research in education, vol 24. American Educational Research Association (AERA), Washington, DC, pp 61–100Google Scholar
  15. Bruck L, Towns M, Bretz S (2008) Characterizing the level of inquiry in the undergraduate laboratory. J Col Sci Teach 38(1):52–58Google Scholar
  16. Business Higher Education Forum (2011) Creating the workforce of the future: the STEM interest and proficiency challenge. Business-Higher Education Forum, Washington, DC.
  17. Cabrera D, Colosi L, Lobdell C (2008) Systems thinking. Eval Prog Plan 31(3):299–310CrossRefGoogle Scholar
  18. Chi M, Deleeuw N, Chiu M et al (1994) Eliciting self-explanations improves understanding. Cogn Sci 18:439–477Google Scholar
  19. Crawford V, Schlager M, Penuel W et al (2008) Supporting the art of teaching in a data-rich, high performance learning environment. In: Mandinach E, Honey M (eds) Linking data and learning. Teachers College Press, New York, pp p109–p129Google Scholar
  20. CSU – Exemplary Online Instruction (2009) The rubric [online]. Available at Accessed 18 May 2018
  21. Cullen R, Harris M, Hill R (2012) The learner-centered curriculum: design and implementation. Jossey-Bass, IndianapolisGoogle Scholar
  22. Daily G, Ehrlich P (1999) Managing earth’s ecosystem: an interdisciplinary challenge. Ecosystems 2:277–280CrossRefGoogle Scholar
  23. DeBari S, Gray K, Monet J (2015) Interactions between water, earth’s surface, and human activity. Accessed 25 June 2018
  24. Deleeuw N, Chi M (2003) Self-explanation: enriching a situation model or repairing a domain model? In: Sinatra G, Pintrich P (eds) Internationalconceptual change. Erlbaum, Mahwah, NJ, pp 55–78Google Scholar
  25. Dodick J, Argamon S, Chase P (2009) Understanding scientific methodology in the historical and experimental sciences via language analysis. Sci Educ 18(8):985–1004CrossRefGoogle Scholar
  26. Edelson D (2001) Learning-for-use: a framework for the design of technology-supported inquiry activities. J Res Sci Teach 38(3):355–385CrossRefGoogle Scholar
  27. Engle R, Nguyen P, Mendelson A (2011) The influence of framing on transfer: initial evidence from a tutoring experiment. Instr Sci 39(5):603–628CrossRefGoogle Scholar
  28. English F (1988) Curriculum auditing. Technomic Publishing, LancasterGoogle Scholar
  29. Fadem C, Shellito C, Walker B (2014) Climate of change: interactions and feedbacks between water, air, and ice. Accessed 25 June 2018
  30. Flavell J (1979) Metacognition and cognitive monitoring: a new area of cognitive-development inquiry. Am Psychol 34(10):906–911CrossRefGoogle Scholar
  31. Ford A (2009) Modeling the environment: an introduction to system dynamics. Island Press, Washington DCGoogle Scholar
  32. Fortner S, Murphy M, Scherer H (2014) A growing concern: sustaining soil resources through local decision making. Accessed 25 June 2018
  33. Foshay A (2000) The curriculum: purpose, substance, practice. Teachers College Press, New YorkGoogle Scholar
  34. Fox B, Rosen J, Crawford M (2008) Distractions, distractions: does instant messaging affect college students’ performance on a concurrent reading comprehension task. CyberPsych Behav 12(1):51–53CrossRefGoogle Scholar
  35. Gagne R, Wager W, Golas K et al (2004) Principles of instructional design, 5th edn. Thomson/Wadsworth, BelmontGoogle Scholar
  36. GETSI (2018) GETSI teaching materials. Accessed 7 June 2018
  37. Gilbert L (1998) Disciplinary breadth and interdisciplinary knowledge production. Knowledge Technol Policy 11(1–2):4–15CrossRefGoogle Scholar
  38. Glatthorn A (1994) Developing a quality curriculum. Association for Supervision and Curriculum Development, AlexandriaGoogle Scholar
  39. Handelsman J, Ebert-May D, Beichner R et al (2004) Scientific teaching. Science 304:521–522CrossRefGoogle Scholar
  40. Hatano G, Oura Y (2003) Commentary: reconceptualizing school learning using insight from expertise research. Educ Res 3(8):26–29CrossRefGoogle Scholar
  41. Harrington J (1970) Ontology of geologic reasoning with a rationale for evaluating historical contributions. Am J Sci 269(3):295–303CrossRefGoogle Scholar
  42. Honebein P (1996) Seven goals for the design of constructivist learning environments. In: Wilson B (ed) Constructivist learning environments: case studies in instructional design. Educational Technology Publications, Englewood CliffsGoogle Scholar
  43. Hurd J (2000) The transformation of scientific communication: a model for 2020. J Am Soc Inf Sci 51(14):1279–1283CrossRefGoogle Scholar
  44. Ivanitskaya L, Clark D, Montgomery G et al (2002) Interdisciplinary learning: process and outcomes. Innov High Educ 27(2):95–111CrossRefGoogle Scholar
  45. Kastens K, Rivet A (2008) Multiple modes of inquiry in earth science. Sci Teach 75(1):26–31Google Scholar
  46. Kastens K, Manduca C, Cervato C et al (2009) How geoscientists think and learn. Eos Trans AGU 90:31CrossRefGoogle Scholar
  47. Krajcik J, McNeill K, Reiser B (2008) Learning-goals-driven design model: developing curriculum materials that align with national standards and incorporate project-based pedagogy. Sci Educ 92(1):1–32CrossRefGoogle Scholar
  48. Lederman N (2007) Nature of science: past, present and future. In: Abell S, Lederman N (eds) Handbook of research in science education. Lawrence Erlbaum, Mahwah, pp 831–879Google Scholar
  49. Libarkin J, Kurdziel J (2006) Ontology and the teaching of earth system science. J Geosci Educ 54(3):408–413CrossRefGoogle Scholar
  50. Linkens G (1999) The science of nature, the nature of science; long-term ecological studies at Hubbard Brook. Proc Am Philos Soc 143(4):558–572Google Scholar
  51. Manduca C, Kastens K (2012a) Geoscience and geoscientists: uniquely equipped to study the earth. In: Earth and mind II: a synthesis of research on thinking and learning in the geosciences, Special paper 486. Geological Society of America, Boulder, pp 1–12Google Scholar
  52. Manduca C, Kastens K (2012b) Mapping the domain of complex earth systems in the geosciences. In: Earth and mind II: A synthesis of research on thinking and learning in the geosciences, Special paper 486. Geological Society of America, Boulder, pp 91–96CrossRefGoogle Scholar
  53. Manduca C, Mogk D (2002) Using data in undergraduate science classrooms: final report on an interdisciplinary workshop at Carleton College. Science Education Resource Center, Carleton College, Northfield, MN. Retrieved from
  54. Midgley G (2008) Response to paper “Systems thinking” by D. Cabrera et al.: the unification of systems thinking: is there gold at the end of the rainbow? Eval Prog Plan 31(3):317–321CrossRefGoogle Scholar
  55. National Research Council (2000) How people learn: brain, mind, experience and school. National Academy Press, Washington DCGoogle Scholar
  56. National Research Council (2001) Grand challenges in environmental sciences. National Academies Press, Washington DCGoogle Scholar
  57. National Research Council (2012) Discipline-based education research: understanding and improving learning in undergraduate science and engineering. The National Academies Press, Washington, DCGoogle Scholar
  58. NOAA (2005) Ocean literacy: the essential principals of ocean sciences K-12, National Geographic Society. National Academies Press, Washington DCGoogle Scholar
  59. Perez A, Schneiderman J, Stewart M, et al (2018) Environmental justice and freshwater resources. Accessed 25 June 2018
  60. Popham W (1997) What’s wrong-and what’s right-with rubrics. Educ Lead 55(2):72–75Google Scholar
  61. Popham W (1999) Where large scale assessment is heading and why it shouldn’t. Educ Meas Issues Pract 18(3):13–17CrossRefGoogle Scholar
  62. Popham W (2008) Transformative assessment. Association for Supervision and Curriculum Development, AlexandriaGoogle Scholar
  63. Pressley M, Borkowski J, Schneider W (1989) Good information processing: what is it and what education can do to promote it? J Exp Child Psychol 43(2):194–211CrossRefGoogle Scholar
  64. Pyle E, Brunkhorst B (2009) Developing and applying the knowledge, skills, and dispositions needed for effective earth science teaching. In: Collins A, Gillespie N (eds) The continuum of secondary science teacher preparation. Sense Publishers, Boston, pp 103–128CrossRefGoogle Scholar
  65. (2014) Course design standards [online]. Available at Accessed 18 May 2018
  66. Ruddiman W (2001) Earth’s climate: past and future. W.H Freeman and Co, New YorkGoogle Scholar
  67. Sayle A (1981) Management audits. McGraw-Hill Book Company, New YorkGoogle Scholar
  68. Schraw G, Moshman D (1995) Metacognitive theories. Educ Psychol Rev 7(4):351–371CrossRefGoogle Scholar
  69. Schraw G, Crippen K, Hartley K (2006) Promoting self-regulation in science education: metacognition as part of a broader perspective on learning. Res Sci Educ 36:111–139CrossRefGoogle Scholar
  70. Stillings N (2012) Complex systems in the geosciences and in geoscience learning. In: In earth and mind II: a synthesis of research on thinking and learning in the geosciences, Special paper 486. Geological Society of America, Boulder, pp 97–112Google Scholar
  71. Taber M, Ledley T, Lynds S et al (2012) Geoscience data for educational use: recommendations from scientific/technical and educational communities. J Geosci Educ 60:249–256CrossRefGoogle Scholar
  72. Trigwell K, Prosser M (1991) Improving the quality of student learning: the influence of learning context and student approaches to learning with learning outcomes. J High Educ 22:251–266CrossRefGoogle Scholar
  73. UCAR and CIRES (2008) Essential principles and fundamental concepts for atmospheric science literacy. UCAR, BoulderGoogle Scholar
  74. USGCRP (U.S. Global Change Research Program) (2009) Climate literacy: essential principles and fundamental concepts. NSF, AlexandriaGoogle Scholar
  75. Virgili C (2007) Charles Lyell and scientific thinking in geology. C R Geosci 339(8):572–584CrossRefGoogle Scholar
  76. Weigold M (2001) Communicating science: a review of the literature. Sci Commun 23(2):164–193CrossRefGoogle Scholar
  77. Wiggens G, McTighe J (2005) Understanding by design. Association for Supervision and Curriculum Development, AlexandriaGoogle Scholar
  78. Wynne B (1991) Knowledges in context. Sci Technol Hum Values 16(1):111–121CrossRefGoogle Scholar
  79. Wysession M, Taber J, Budd D et al (2009) Earth science literacy: the big ideas and supporting concepts of earth science. NSF, AlexandriaGoogle Scholar
  80. Young A, Fry J (2008) Metacognitive awareness and academic achievement in college. J Scholarsh Teach Learn 8(2):1–10Google Scholar
  81. Zeegers P (2001) Approaches to learning in science: A longitudinal study. Br J Educ Psychol 71(1):115–132CrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • David Steer
    • 1
    Email author
  • Ellen R. Iverson
    • 2
  • Anne E. Egger
    • 3
  • Kim A. Kastens
    • 4
  • Cathryn A. Manduca
    • 2
  • David McConnell
    • 5
  1. 1.Department of GeosciencesThe University of AkronAkronUSA
  2. 2.Science Education Resource Center, Carleton CollegeNorthfieldUSA
  3. 3.Geological Sciences and Science EducationCentral Washington UniversityEllensburgUSA
  4. 4.Marine Geology and Geophysics, Lamont-Doherty Earth ObservatoryColumbia UniversityNew YorkUSA
  5. 5.Department of Marine, Earth, and Atmospheric SciencesNorth Carolina State UniversityRaleighUSA

Personalised recommendations