Abstract
Effective assessments offer more than just a measurement of learning. When sequenced and aligned to evidence-based curriculum, assessments can facilitate learning and inform instruction. The Interdisciplinary Teaching about Earth for a Sustainable Future (InTeGrate) project viewed assessment as a core strategy of the curriculum being developed. InTeGrate used a community-based approach to assessment. An assessment team engaged with materials development teams to facilitate module and course assessment design, developed common instruments to assess student learning, analyzed student data from pilot tests to inform curriculum revisions, and used the findings to fine-tune assessment instruments and to shed light on opportunities for faculty development within the geoscience education community. The community approach ensured high-quality curricula and assessments and strategies to investigate learning at the module, course, and project level. Perhaps more importantly, the approach increased the capacity of the geoscience education community by harnessing the expertise from within the community to promote a culture of evaluative thinking.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
AAAS (1993) Benchmarks for science literacy. American Association for the Advancement of Science, Washington, DC
AAAS (2011) Vision and change in undergraduate biology education: a call to action. American Association for the Advancement of Science, Washington, DC
American Geosciences Institute (2009) Status of the geocience workforce. https://www.americangeosciences.org/workforce/reports. Accessed 9 Mar 2018
Argyris C (1976) Single-loop and double-loop models in research on decision making. Adm Sci Q:363–375
Blaschke LM (2012) Heutagogy and lifelong learning: a review of heutagogical practice and self-determined learning. Int Rev Res Open Dist Learn 13(1):56–71
Boix Mansilla V, Duraising ED (2007) Targeted assessment of students’ interdisciplinary work: an empirically grounded framework proposed. J High Educ 78(2):215–237
Bybee R, McCrae B, Laurie R (2009) PISA 2006: an assessment of scientific literacy. J Res Sci Teach 46(8):865–883
Champagne AB, Kouba VL (2000) Writing to inquire: written products as performance measures. In: Mintzes JJ, Wandersee JH, Novak JD (eds) Assessing science understanding: a human constructivist view. Elsevier, Burlington, pp 224–246
Egger AE, Kastens KA, Turrin MK (2017) Sustainability, the Next Generation Science Standards, and the education of future teachers. J Geosci Educ 65(2):168–184
Egger AE, Bruckner MZ, Birnbaum SJ, Gilbert LA (this volume) Facilitating the development of effective interdisciplinary curricular materials. In: Gosselin DC, Egger AE, Taber JJ (eds) Interdisciplinary teaching about earth and the environment for a sustainable future, Association of Environmental Sciences and Studies (AESS) book series. Springer, Cham
Fuhrman M (n.d.) Questionnaire to measure indicators for recruitment/retention in geoscience careers, developed to evaluate projects in NSF’s Opportunities for Enhancing Diversity in the Geosciences (OEDG) program. American Institutes for Research
Gilbert LA, Gross D, Kreutz K (2016) Systems thinking. https://serc.carleton.edu/integrate/teaching_materials/syst_thinking/index.html. Accessed 1 Mar 2018
Gilbert LA, Iverson ER, Kastens KA, Awad A, McCauley EQ, Caulkins JL, Steer DN, McConnell DA, Manduca CA (2017) Explicit focus on systems thinking in InTeGrate materials yields improved student performance. Paper presented at the GSA Annual Meeting, Seattle, WA, 22–25 October 2017
Gosselin DC, Manduca CA, Bralower TJ, Egger AE (this volume) Preparing students to address societally relevant challenges in the geosciences: the InTeGrate approach. In: Gosselin DC, Egger AE, Taber JJ (eds) Interdisciplinary teaching about earth and the environment for a sustainable future, AESS Interdisciplinary Environmental Studies and Sciences series. Springer, Cham
Houlton HR (2010) Geoscience major career choice: investigation of pathways that lead students to the geosciences. Paper presented at the 2010 GSA Annual Meeting, Denver, CO
InTeGrate (2012a) InTeGrate assessment team 2012 meeting notes.
InTeGrate (2012b) Your module/course development timeline. https://serc.carleton.edu/integrate/info_team_members/currdev/timeline.html. Accessed 9 Mar 2018
InTeGrate (2014) Using InTeGrate modules and courses. https://serc.carleton.edu/integrate/teaching_materials/using/index.html. Accessed 12 Jan 2018
InTeGrate (2015) Developing systems thinking webinar. https://serc.carleton.edu/integrate/workshops/webinars/2015_2016/systems.html. Accessed 9 Mar 2018
InTeGrate (2017a) Classroom data collection and grading tools. https://serc.carleton.edu/integrate/about/data_tools.html. Accessed 9 Mar 2018
InTeGrate (2017b) Implementation programs. https://serc.carleton.edu/integrate/programs/implementation/index.html. Accessed 9 Mar 2018
InTeGrate (2018a) Assessment and project evaluation. https://serc.carleton.edu/integrate/about/assessment.html. Accessed 9 Mar 2018
InTeGrate (2018b) Essay assessments: interdisciplinary and systems thinking. https://serc.carleton.edu/integrate/about/essays.html. Accessed 9 Mar 2018
InTeGrate (2018c) Faculty surveys. https://serc.carleton.edu/integrate/about/faculty_surveys.html. Accessed 9 Mar 2018
InTeGrate (2018d) Geoscience literacy exam (GLE). https://serc.carleton.edu/integrate/about/gle.html. Accessed 9 Mar 2018
InTeGrate (2018e) HBCU Working Group. https://serc.carleton.edu/integrate/about/hbcu_working_group.html. Accessed 9 Mar 2018
InTeGrate (2018f) The InTeGrate attitudinal instrument (IAI). https://serc.carleton.edu/integrate/about/iai.html. Accessed 9 Mar 2018
InTeGrate (2018g) Teach systems thinking. https://serc.carleton.edu/integrate/teaching_materials/themes/expert_thinking/systems.html. Accessed 9 Mar 2018
Janzen R, Ochocka J, Turner L, Cook T, Franklin M, Deichert D (2017) Building a community-based culture of evaluation. Eval Program Plann 65:163–170
Johnson R, Snow J, Abshire W, Buhr S, Cullen H, Denning S, Holland M, Manduca C, Pennington P, Schoedinger S, Schultz P, Shepherd JP (2009) Essential principles and fundamental concepts for atmospheric science literacy. NSF: Geoscience Education Program
Kastens KA, Manduca CA (2017) Using systems thinking in the design, implementation, and evaluation of complex educational innovations, with examples from the InTeGrate project. J Geosci Educ 65(3):219–230
Kastens KA, Manduca CA (2018) Leveraging the power of community of practice to improve teaching and learning about the Earth. Change: The Magazine of Higher Learning
Kastens KA, Mara V (2016) InTeGrate evaluation report: underrepresented minorities
Kastens KA, Mara V (2017) InTeGrate evaluation report: IAI responses of InTeGrate versus non-InTeGrate students
Kastens KA, Mara V (2018) Where is InTeGrate having its impact on Earth and environmental motivation and action? An analysis of IAI data with respect to student’s reason for taking the course
Labin SN, Duffy JL, Meyers DC, Wandersman A, Lesesne CA (2012) A research synthesis of the evaluation capacity building literature. Am J Eval 33(3):307–338
LaDue ND, Clark SK (2012) Educator perspectives on Earth system science literacy: challenges and priorities. J Geosci Educ 60(4):372–383
Libarkin JC, Anderson SW (2005) Assessment of learning in entry-level geoscience courses: results from the Geoscience Concept Inventory. J Geosci Educ 53(4):394–401
Manduca CA, Iverson ER, Luxenberg M, Macdonald RH, McConnell DA, Mogk DW, Tewksbury BJ (2017) Improving undergraduate STEM education: the efficacy of discipline-based professional development. Sci Adv 3(2):e1600193
National Academies of Sciences, Engineering, and Medicine (2016) Science literacy: concepts, contexts, and consequences. The National Academies Press, Washington, DC. https://doi.org/10.17226/23595
NOAA, National Geographic Society, COSEE, National Marine Educators Association (2005) Ocean literacy: the essential principals of ocean sciences K-12. https://oceanservice.noaa.gov/education/literacy.html. Accessed 9 Mar 2018
Novak JD, Mintzes JJ, Wandersee JH (2000) Learning, teaching, and assessment: a human constructivist perspective. In: Mintzes JJ, Wandersee JH, Novak JD (eds) Assessing science understanding: a human constructivist view. Elsevier, Burlington, pp 1–13
On the Cutting Edge (2016) About the Cutting Edge embedded assessment project. https://serc.carleton.edu/NAGTWorkshops/embedded_assessment/index.html. Accessed 1 Mar 2018
Orr CH, McDaris JR (this volume) Supporting implementation of program-level changes to increase learning about Earth. In: Gosselin DC, Egger AE, Taber JJ (eds) Interdisciplinary teaching about earth and the environment for a sustainable future, Association of Environmental Sciences and Studies (AESS) book series. Springer, Cham
Patton MQ (2010) Developmental evaluation: applying complexity concepts to enhance innovation and use. Guilford Press, New York
PISA (2015) Program for international student assessment, oecd.org
Preskill H, Boyle S (2008) A multidisciplinary model of evaluation capacity building. Am J Eval 29(4):443–459
Rockman (2013) On the Cutting Edge project: year 3 independent evaluation report. http://serc.carleton.edu/files/NAGTWorkshops/about/2013_external_evaluation_repor.pdf. Accessed 10 Mar 2018
SERC (2017) Serkit. https://serc.carleton.edu/serc/about/serckit.html. Accessed Dec 2017
Steer D, Iverson ER, Manduca CA (2013) Piloting a geoscience literacy exam for assessing students’ understanding of Earth, climate, atmospheric and ocean science concepts. Paper presented at the AGU Fall Meeting Abstracts, San Francisco, CA, 9–13 December 2013
Steer D, Iverson ER, Egger AE, Kastens KA, Manduca CA, McConnell D (this volume) The InTeGrate materials development rubric: a framework and process for developing curricular materials that meet ambitions goals. In: Gosselin DC, Egger AE, Taber JJ (eds) Interdisciplinary teaching about Earth and the environment for a sustainable future, Association of Environmental Sciences and Studies (AESS) book series. Springer, Cham
U.S. Global Change Research Program (2009) Climate literacy: essential principles and fundamental concepts. National Oceanic and Atmospheric Administration and the American Association for the Advancement of Science
Viskupic K, Steer D, Kortz KM, Perkins D, Wirth K, Herbert B, Singer J, Manduca CA (2014) Monitoring undergraduate growth through the major using embedded assessment questions, Geological Society of America Abstracts with Programs, Vancouver, BC 46(6)
Wiggins G, McTighe J (1998) Understanding by design. Association for Supervision and Curriculum Development, Alexandria
Wysession ME (2012) Implications for earth and space in new K–12 science standards. Eos Trans Am Geophys Union 93(46):465–466
Wysession M, Taber J, Budd DA, Campbell K, Conklin M, LaDue N, Lewis G, Raynolds R, Ridky R, Ross R, Tewksbury B, Tuddenham P (2009) Earth science literacy: the big ideas and supporting concepts of Earth science. NSF: The Earth Science Literacy Initiative
Wysession ME, LaDue N, Budd DA, Campbell K, Conklin M, Kappel E, Lewis G, Raynolds R, Ridky RW, Ross RM (2012) Developing and applying a set of earth science literacy principles. J Geosci Educ 60(2):95–99
Acknowledgment
Leads
David Steer, University of Akron Main Campus (2011–2017)
Ellen Iverson, Science Education Resource Center, Carleton College (2011–2017)
Stuart Birnbaum, University of Texas at San Antonio (2012–2017)
Team Members
Leilani Arthurs, University of Nebraska at Lincoln (2013–2017)
Aida Awad, Broward Community College (2012–2017)
Barbara Bekken, Virginia Polytechnic Institute and State University (2013–2017)
Joshua Caulkins, University of Rhode Island (2012–2017)
Wendy J. Harrison, Colorado School of Mines (2012)
Megan Plenge, University of North Carolina at Chapel Hill (2014–2017)
Sian Proctor, Maricopa County Community College (2012)
Mary Savina, Carleton College (2012–2017)
Susan Sullivan, University of Colorado at Boulder (2012–2017)
Emily Geraghty Ward, Rocky Mountain College (2012–2014)
Karen Viskupic, Boise State University (2012–2017)
Project Liaison
Kristin O’Connell, Science Education Resource Center, Carleton College
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Appendix: The Geoscience Literacy Exam: 8 Question Instrument (GLE Common-8)
Appendix: The Geoscience Literacy Exam: 8 Question Instrument (GLE Common-8)
Literacy addressed | Question |
|---|---|
Earth science literacy Natural hazards pose risks to humans | 1. Natural hazards can be put into two major categories. Some natural hazards can be made worse by humans; others are largely independent of human activities. Select the natural hazard least likely to be affected by human activity (a) Forest fires (b) Tsunami (c) Landslides (d) Coastal erosion |
Earth science literacy Earth is continuously changing | 2. Which of the following geologic processes are most likely caused by the interactions between the tectonic plates at their boundaries? Select all that apply (a) Earthquakes (b) Continental glaciation (c) Floods (d) Volcanic eruptions (e) Mountains |
Ocean literacy The ocean supports a great diversity of life and ecosystems | 3. Which of the following statements about the distribution of life in the oceans is most correct? (a) Life is more abundant and diverse in some parts of the ocean than in others (b) Life is abundant and diverse throughout the ocean (c) Life is less abundant and diverse in the oceans than it is on land |
Ocean literacy The ocean and humans are inextricably interconnected | 4. Which of the following ways do humans affect oceans? Select all that apply (a) Humans alter ocean ecosystems through fishing (b) Humans alter shorelines through development (c) Humans mine mid-ocean ridges (d) Humans change overall ocean composition by desalination (e) Humans alter tidal cycles |
Atmospheric literacy Earth’s atmosphere continuously interacts with other components of the Earth System | 5. Which of the following processes primarily involves the atmosphere and the biosphere? (a) The formation of limestone (b) The photosynthetic cycle (c) The hydrological cycle |
6. Which of the following processes are sources of carbon to the atmosphere? Select all that apply (a) Plant decay (b) Limestone formation (c) Cattle ranching (d) Fossil fuel use | |
Climate literacy Our understanding of the climate system is improved through observations, theoretical studies, and modeling | 7. There are several climate models used to research future change. Which climate modeling statement about twenty-first-century temperature change projections is most accurate? (a) Climate model projections do not agree on future likely outcomes (b) Climate model projections show similar trends for future outcomes (c) Climate model projections show the same results for future outcomes |
8. The first reasonably accurate mercury thermometers were invented in 1724, almost 300 years ago. What kinds of processes and/or data are used by scientists to determine temperatures more than 10,000 years in the past? Select all that apply (a) Written records (b) Ice cores (c) Tree rings (d) Sedimentary layers (e) Oxygen isotopes |
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this chapter
Cite this chapter
Iverson, E.R., Steer, D., Gilbert, L.A., Kastens, K.A., O’Connell, K., Manduca, C.A. (2019). Measuring Literacy, Attitudes, and Capacities to Solve Societal Problems. In: Gosselin, D., Egger, A., Taber, J. (eds) Interdisciplinary Teaching About Earth and the Environment for a Sustainable Future. AESS Interdisciplinary Environmental Studies and Sciences Series. Springer, Cham. https://doi.org/10.1007/978-3-030-03273-9_5
Download citation
DOI: https://doi.org/10.1007/978-3-030-03273-9_5
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-03272-2
Online ISBN: 978-3-030-03273-9
eBook Packages: Earth and Environmental ScienceEarth and Environmental Science (R0)