Introduction

In spring, summer, and fall of 2022, as elementary school students across the USA returned to in-person classrooms after months of stay-at-home pandemic education, children’s math achievement scores fell dramatically in just about every state (Kuhfeld, Soland, Lewis, & Morton, 2022; Mervosh & Wu, 2022). As educators struggled with how to respond instructionally, we decided to bring back in a redesigned format an online math tutoring system that had its first iteration years earlier and then lapsed in use.

Our newly launched system, Usable Math (https://usablemath.org/), provides students and teachers with 3rd, 4th, and 5th grade-level word problems from publicly released standardized state math tests together with math problem-solving strategies presented by four online math friends or coaches: Estella Explainer, Chef Math Bear, How-to-Hound, and Visual Vicuna (Fig. 1).

Fig. 1
figure 1

A screenshot from UsableMath.org showing the four math coaches and their strategies

Full size image

Usable Math’s math coaches offer words, pictures, gifs, charts, and animations to provide students with multiple alternative approaches to math problem solving: reading comprehension (Estella Explainer), number operations (Chef Math Bear), strategic thinking (How-to-Hound), and visually illustrated learning (Visual Vicuna). After students apply the coaching strategies to help them solve a problem, the system shows a slide with a growth mindset–building statement and humorous image intended to promote student enjoyment, self-confidence, and continuous learning. We made Usable Math an open education resource freely accessible by anyone with an Internet connection.

To date, we have released modules for learning area and perimeter, division, fractions, algebraic thinking, rounding, estimation, place value, add and take away, money, data and graphs, lines and lines of symmetry, grids and ordered pairs, shapes and angles, decimals, and measurement based on topics from the Massachusetts Mathematics Curriculum Framework as well as the nationwide Common Core Standards for Mathematics (2022). We have also developed three additional modules focusing on math and history, math and story writing, and math and citizen science.

As a title, the word Usable can be read and pronounced in three ways:

  • U Able meaning you (every young math learner) can be a math problem solver.

  • Us Able meaning together all of us (students, teachers, and family members in classrooms and homes) can be math problem solvers.

  • Usable meaning anyone (young or older) is able to develop their math problem-solving skills with practice, effort, and support from technology tools and online coaches.

Research on Tutoring and Online Tutors

The goal of tutoring is to individualize or differentiate instruction for students. Tutoring allows individuals and small learning groups to preview, review, or learn material at their own pace, with or without adult supervision or direction. Studies have found that students receiving online tutoring show gains in attitude toward and confidence in learning as well as improved academic skills. Researchers in Spain and Italy recently reported gains in math test scores among disadvantaged secondary school youth who received online tutoring from adult math teachers or volunteer university students (Gortazar, Hupkua, & Roldán, 2022; Carlana & La Ferrara, 2021).

High-impact tutoring is the focus of the National School Support Accelerator Project, a large-scale online tutoring initiative, being conducted by Brown University, now underway at sites around the USA. High-impact tutoring, whose features include three or more sessions a week, strong student-tutor relationships, formative assessments, and alignment with school curriculum, has been shown to generate positive learning gains for students (Robinson & Loeb, 2021). High-impact tutoring is intended to supplement, not replace, ongoing classroom learning. While researchers have not fully investigated how each element of high-impact tutoring programs function pedagogically, gains for students appear to result from a combination of positive connections between students and tutors, making the tutoring fit the needs and interests of the students, and use of new approaches instructional approaches during tutoring sessions — all goals we have sought to achieve in our Usable Math site design.

High-impact tutoring is also known as “high-dosage” tutoring, and for-profit education companies in the USA have marketed it as an online service that school systems can purchase for students and families to use. But researchers found few students actually used these programs, and students who were struggling with math learning were the least likely to opt-in for assistance (Robinson, Bisht, & Loeb, 2022). More than one observer has commented on the irony of using online tutors to address learning losses by students during pandemic era online learning. We too recognized this dynamic and have sought to incorporate features to motivate and engage students in our Usable Math system design. We are encouraged by the positive gains reported in a study where college students from universities around the USA provided online tutoring to middle school students in Chicago (Kraft, List, Livingston, & Sadoff, 2022).

Web-based tutoring systems have achieved positive learning outcomes for students (Beal, et al. 2007; Maloy, Edwards, & Anderson, 2010). Computer science researchers at the University of Massachusetts Amherst have worked on the development of intelligent math tutoring systems that adapt computer responses to student learning needs and emotions (Woolf & Arroyo, 2015; Woolf, et al., 2014). More recently, these researchers have been investigating wearable technologies to promote computational thinking and STEM learning for 5–12th grade students through mobile technology–augmented active game play and game creation (Arroyo, et al., 2022). While not involved in building intelligent tutoring systems, our work is focused on open access web-based systems to support math learning for elementary school-age learners.

Within tutoring systems, researchers have asserted the importance of providing students with different types of supports as a way to differentiate math learning for students, particularly in the case of word problems. Such learning supports can include changing how a problem is presented, helping students understand the language of a problem, and showing more than one approach to solving a problem (Russo, Minas, Jewish & McCosh, 2020). Researchers in Germany have found that while teachers seek ways to differentiate learning for students, many chose activities different from and more calculation driven than would task-design experts (Brady, Holzapfel, & Leuders, 2021). In designing Usable Math, we have sought to differentiate supports for students through the hints provided by the four coaches in Google Slides.

Replacing and Redesigning an Older Online Tutoring System

Usable Math replaced an original older system called 4MALITY (4-Coach Mathematics Active Learning Tutoring sYstem) that had been housed on a shared computing resource at our university, built with the earlier versions of HTML and animation software, and while free, required registering to use (Razzaq, et al., 2011). 4MALITY was developed in part with funding support from the Verizon Foundation and a grant from the US Department of Education, Institute of Education (IES). The system was in use from 2005 to 2012 when grant funding ended.

4MALITY did show promise as a tutoring system. In one study, fourth graders in two Massachusetts communities — one small rural city and a small suburban district — who completed 11 modules showed an average of 23.5% gain from pretest to post-test across all modules. Girls slightly outgained boys while students below grade level in mathematics performed as well as on-grade level students (Maloy, et al., 2014a).

However, the 4MALITY system proved difficult to maintain. Adding problems to the system and making changes to existing questions required HTML coding skills and took considerable time to do. Maintaining a SQL database of new and existing users also took time away from system improvements. Teaching responsibilities at our university shifted and coupled with the loss of grant funding, the system was no longer maintained after 2102. The shift to emergency remote teaching during the COVID pandemic as well as the emergence of more and more powerful free-to-access Google tools, including Google Slides, gave us the opportunity to redesign the older 4MALITY system.

We began by rebranding 4MALITY as Usable Math while retaining learning goals from our previous system, including:

  • Teaching mathematics concepts and problem-solving skills based on the Massachusetts Mathematics Curriculum Framework (2017) and the Common Core State Standards for Mathematics

  • Integrating strategies from four coaches, Estella Explainer, Chef Math Bear, How to Hound, and Visual Vicuna (see Fig. 1), who offer reading, computation, strategy, and visually comprehension strategies for solving math problems

  • Inviting learning by students with different understandings and math knowledge

  • Helping students prepare for standardized math tests with practice questions that follow a multiple choice or open response style

  • Providing students with continuous feedback about their performance

  • Offering ways for teachers to assess and document student’s accomplishments in math learning

To make the new platform accessible and easier to use, we created Usable Math as an online browser-based system with no reliance on any additional software installation. Users can quickly access the learning system from any popular internet browsers, such as Mozilla Firefox, Apple Safari, or Google Chrome. We also designed the website’s interface to be easy to navigate and based on the design principles of simplicity, clarity, and consistency.

The new system is designed to meet all accessibility requirements of digital devices and continue the learning goals of 4MALITY. We reuse the practice questions that follow a multiple choice or open response style and illustrate them with some of the original and many brand new strategies. A core purpose of the earlier system that we adhered to in the new system is inviting students to “think” as a math problem solver rather than answering questions as quickly as they can to speedily finish modules without consulting any strategies. We wanted students to become comfortable with deliberate and intentional thought before answering a question by consulting coaches and examining word problems from different problem-solving points of view. We call this perspective “thinking and checking” or “thinking like a detective” to highlight how students can practice new behaviors and gain confidence as solvers of complex word problems.

As we designed the Usable Math platform, we sought feedback from multiple users in the form of first-use reactions to the site from teachers and students. This was an informal process where we observed and talked with individuals and small groups as they viewed the site and completed different problem-solving modules. At this stage of development, we felt we were not ready to collect formal survey data, although a survey response form has been posted at the end of every module. Instead, we focused on basic design-related questions: Was the site easy to navigate? Were slides for problems visually engaging and mathematically accurate? What additional topics or problems did teachers and students want on the site?

To get reactions from student users, we applied for and earned a small classroom grant from the MassCUE (Massachusetts Computer Using Educators) organization to purchase seven iPads for students to use in accessing the Usable Math website. Students in a fourth grade classroom in a nearby elementary school used the system first on Chromebooks and then on iPads. Students gave feedback to the teacher who relayed that information to us to use in module design and redesign. We got additional student feedback from fourth and fifth graders in summer 2023 during a series of weekly math learning workshops at a local Boys and Girls Club where students played math games and reviewed Usable Math modules. We organized the workshops and gathered feedback through individual and small group conversations with students.

To get feedback from teachers, we led two professional development workshops for 20 teachers who provided ideas and suggestions. College students in a course we teach at our university also reviewed modules during class workshops and provided feedback to us. Feedback was integrated into the following system design decisions that we discuss next.

Design Decisions and Instructional Choices for Teachers

Our goal in designing Usable Math was to make the overall learning system more accessible and engaging for the students and teachers alike while providing learners with multiple approaches to math problem solving. As we developed our new site, we realized that we had to make key system design decisions about the form and function of our tutor, and that teachers in classrooms will face similar choices for themselves and for students when engaging in online tutoring for math learning. Here, we describe those design decisions, focusing on the choices we made and the choices teachers will face in their own educational settings.

Design Decision One: Four Math Coaches Offer Multiple Perspectives

In Usable Math, we chose to retain four coaches featuring different approaches to math problem solving as a focal point of classroom instruction and student learning. Many students and teachers think of math as a singular word, meaning one way — a correct procedure — exists for solving math problems. We use the system to demonstrate multiple approaches to math problem solving; hence, everyone has the opportunity to experience word problems as puzzles that can be solved in more than one way.

Helping students expand their problem solving and computational learning skills by utilizing more than one approach to solve a math word problem is supported by the coaches’ varied ideas and suggestions (Table 1).

Table 1 Diverse problem-solving approaches offered by usable math’s virtual coaches
Full size table

Teachers in math classrooms will face choices about how to introduce multiple problem-solving perspectives such as those presented in Usable Math as they teach problem solving and computation to young learners. Those choices exist whether teachers have students use online or paper-based materials. Presenting math from multiple viewpoints can be a time-intensive approach since students will need to practice different problem-solving strategies. In many US classrooms, math now follows a scripted curriculum that demands teachers and students cover specific amounts of material each week and move on to new material the next week in preparation for standardized math achievement tests. The pace of a scripted curriculum may make it difficult to find the time to introduce different problem-solving approaches such as those presented in Usable Math. We envision Usable Math as a way for teachers to expand the problem-solving skill of students, but it will require finding time to utilize the system within the pace of the regular school day schedule.

Design Decision Two: a Google Slides-based Platform

To develop a “learning tool” delivering math content and acting as frontend resource for children, the Usable Math researchers began by experimenting with various slideshow software available on the internet. We reviewed a variety of slide-based presentation software before deciding on Google Slides. Our candidates included Microsoft Powerpoint, Apple Keynote, Canva, and Prezi. The decision to select Google Slides arose from a few key factors: (1) the first is price. Google Slides is a free format; neither the teacher, the learner, nor the developers pay any price to view and develop content; (2) the second is the learning curve associated with creating Google Slides — the interface is extremely intuitive and straightforward, enabling even the most novice of users to create and share presentations in little time and without prior knowledge of design or coding skills; (3) the third is reliability: Google Slides is consistently online, giving our Usable Math team the advantage of being able to instantly update and publish math modules without delays; (4) no registration is required to use the system enabling users and designers the freedom to share it with anyone and for any purpose; and finally, (5) Cross-platform compatibility: Google Slides can be used on various devices and operating systems, making it a versatile choice for our project.

In contrast, the original 4MALITY system was developed using programming languages such as HTML, CSS, Java, and SQL. Simple tasks such as authoring new problems and modifying math modules needed technical expertise and use of computing server resources. Not only do Google Slides make authoring quick, easy, and free, but math problems in Usable Math are embedded in modules based on mathematical learning standards and when presented in Slideshow mode are easily viewed on desktops, laptops, tablets, and smartphones. Google Slides allows researchers to conceptualize math word problems and make them presentable in a standardized format. Google Slides as the platform for the new system offers flexible ways to author and display problems, embed animations and gifs, and present information invitingly.

Teachers in math classrooms also have choices about what system to use to deliver content to students, including what online materials to use as part of math instruction time. Some of the learning materials teachers must use are decided for them by the curriculum purchased by school districts, but most teachers have the option to develop their own materials as well. Google Slides offers teachers user-friendly features that make it easy to create curriculum materials specifically tailored to the needs of the students in a classroom. Since Usable Math is an open resource, teachers can also make a copy of one of our slideshows and then recreate and remix it any way they want.

Design Decision Three: Click-to-See Method and User Control of Slides

We chose a click-to-see method as part of Usable Math design so users can control what happens during each learning experience, whether it is teachers leading classroom presentations, students learning independently or collaboratively on computing devices, or families or after-school programs utilizing the system outside of school time. In this click-to-see method, users reveal parts of each math problem one step at a time, getting new information to appear on the screen with each click of their mouse or return key. Each user chooses how quickly or slowly they access information. The intent of click-to-see is to engage both learners and teachers in viewing each coach and analyzing their strategy hints before viewing the answer choices and choosing one. The goal is for users to choose a pace, slow down, and think through each problem before answering it.

A click-to-see method seeks to develop an alternative to practiced routines in many classrooms where students think briefly about what a correct answer might be and make their choice before considering and synthesizing all the information they need to solve a word problem. Paper worksheets or practice tests, especially timed tests, encourage the fast completion of math problems instead of a focused investigation. Paper or online worksheets are endless practice methods in schools. Many students decide that doing them fast is a way of moving on to other, perhaps less boring activities. Timed tests support not thinking but memorizing responses for speed. What these accomplish is the opposite of problem solving.

Here is one way click-to-see can work step-by-step in Google Slides with a teacher displaying a math word problem and the Usable Math coaches on a classroom whiteboard or screen for children to see.

  • A word problem appears on the screen without displaying answer choices or coaching strategies. Students and teacher(s) together converse about the problem to consider what it is asking and how thinking mathematically might offer a solution to the question (see Fig. 2).

  • The facilitator, either a student or the math teacher, clicks once and the slideshow displays Estella Explainer’s hint which offers a reading coaching strategy to solve the presented math problem. The original math problem continues to display at the top of the screen, while each hint appears in the bottom section of the screen. At this point, the classroom is engaged in actively conversing about the problem from the lens of Estella Explainer’s scaffolding hint.

  • Next, someone clicks again and Chef Math Bear shows a different computational coaching strategy.

  • With another click, How-to-Hound presents a strategic coaching strategy.

  • With another click, Visual Vicuna complements other coaches’ clues with a movie, chart, graph, or picture as a visual coaching strategy (Fig. 3).

  • Students have opportunities to see, read, and think about the problem with solving ideas from the coaches before another click reveals the four answer choices, but does not indicate the correct one. Now, students have one more opportunity to analyze and discuss with one another what they know and have learned from the coaches to help answer the question.

  • A final click highlights the solution to the problem among the answer choices.

  • Before continuing on to the next word problem, a surprising, funny motivational statement and/or gif appears on a slide to offer encouragement and affirmation to students.

Fig. 2
figure 2

A screenshot of one of the math word problems from the division math module

Full size image
Fig. 3
figure 3

Screenshot showing four hints on a math word problem

Full size image

In our Usable Math design, we decided that students would go through click-to-see hints from coaches before they could answer each question. In the earlier 4MALITY system, students could simply answer the question without accessing the hints. 4MALITY provided information, choices, visuals, and animations, but if the learner did not want to look at any of the information, they had the power to ignore it all and simply answer questions one after the other. As a result, many students, acting on the belief that being smart means solving problems as fast as you can, would choose to ignore the coaches entirely and rush ahead to answer the questions. By not taking the time to carefully read each word problem and think through problem-solving strategies, students made mistakes and practiced erroneous skills. This dynamic is not an easy one to combat. Students need to have a choice to feel as though they are in charge of their learning, but sometimes, that may mean they are going to click through the slides in Usable Math, ignore the hints, and find the answer.

Teachers in math classrooms also face choices about how quickly or slowly they want students to access hints and information for solving word problems, whether they are using Usable Math or a tutoring/teaching system of their own design. Researchers from Australia have argued that children need to have the choice to access or not access math problem-solving prompts and that teachers have the responsibility to build classroom cultures where seeking help before answering questions is favorably regarded as what skilled math learners do when encountering math problems (Russo, Minas, Jewish, & McCosh, 2020). We agree that teachers should keep encouraging students to consult the coaches while giving youngsters the freedom to click past a coach if they are confident they have the answer to the problem. Getting students to develop a problem-solving mindset where viewing coaches before answering will take time and practice. Each classroom teacher needs to create their own ways to support student thinking, viewing, practicing, and gaining coaches’ ideas.

Design Decision Four: Growth Mindset Statements to Motivate Students

As a new feature in Usable Math, we decided to include a growth mindset–building motivational slide after every problem as a way to encourage young students to continue using the system for learning. Each of these slides is intended to give feedback and encouragement (Fig. 4). Examples include cartoon characters giving each other high fives along with the statement, “The coaches are your TEAM!” or a dancing cat announcing “You got MATH moves!” Later, in a module as a boy and his dog paint a fence appears the statement, “The more you do, the more you learn.” When 5 problems are finished in one module, a girl holds up her rainbow-colored 5 fingers in celebration of this accomplishment.

Fig. 4
figure 4

Screenshots showing growth mindset and affirmative statements in Usable Math, including a reference to Dug from the Hollywood movie “Up” (Walt Disney Studios, 2009)

Full size image

In writing growth mindset–building statements, we have incorporated the insights of researchers (Boaler, 2015; Dweck, 2007) who urge adults to use language that emphasizes students’ effort, hard work, persistence, and belief in themselves as competent learners. We do not use the same statements within a module so youngsters can anticipate a surprise affirmation after each word problem they do. Motivational statements convey a belief that youngsters can learn all math with the right tools, the right beliefs, the right coaches, and their own interest, focus, and practice.

Teachers in math classrooms have choices about what to say when responding to the problem-solving efforts of students. Psychologist Alfie Kohn (2021) famously urged adults to stop saying “Good Job!” when responding to what children are doing academically or creatively. Broad but brief praise statements, he believed, actually reduce motivation and achievement as youngsters seek validation from adults rather than developing a personal sense of accomplishment from their efforts and self-expressions. Whether agreeing or not with Kohn’s claims, every teacher must decide what to say and when to say it as students engage in math learning. They can use statements directly from Usable Math, modify our affirmations, or design their own mindset-builders with pictures, text, and direct interactions. However they respond, their language choices may well be a crucial ingredient sustaining youngsters’ interest in solving math problems.

Design Decision Five: Students as Math Question Writers

As we initially designed Usable Math, we did not fully consider that students might write their own math problems (with answer choices and problem solving strategies) as part of math learning. But after students in one 4th grade, with encouragement from their teacher, began writing their own problems, we started to include problem design invitations for students in our modules. Figure 5 is a screenshot of a Google Slide inviting students to do their own math writing after engaging with one of the learning modules in Usable Math, in this case a forthcoming geometry module.

Fig. 5
figure 5

Screenshot showing the math hint design template

Full size image

Encouraging students to write their own math problems made sense academically. Our research has focused on how self-chosen creative writing propels student learning (Edwards, et al., 2020). Researchers have documented how writing offers ways for students to record, consider, and revise their thinking about math problem solving (Burns, 2015; Whitin & Whitin, 2000). Putting words and numbers on paper and/or screens promotes actively creating a word problem for a concept, crafting answer choices to be tricky to decide between, and reflecting understanding of writing math problems and solving them. Youngsters are not just memorizing number operations or mathematical terms, but describing how those concepts can be utilized in solving word problems.

Fourth graders expanded their creative math question writing, moving on from authoring their own math problems to authoring hints based on what they imagine Estella Explainer, Chef Math Bear, How-to-Hound, and Visual Vicuna would say to also inventing entirely new characters and short math stories to include in their questions (Fig. 6). Some students added their own growth mindset statements after each of the problems and then assembled their questions in short slideshows for classmates to view.

Fig. 6
figure 6

Screenshot of student-created math word problem including a hint

Full size image

The student text edited for ease of reading is made available below:

Mr. Man has a collection of pencil sharpeners shaped like cats. Mr. Man has 47 of them. He goes to Pencil Sharpeners Shaped Like Cats Mini Mall every Tuesday. Every time he goes there he buys 22 more. How many sharpeners does he have after 9 weeks?

A. 236 B.244 C. 298 D. 245 E. 239

(Next to a picture of Kermit the Frog, the student wrote) “First do 9 times 22 then add it to the 47”

Teachers in math classrooms have choices about ways that elementary school students can take on roles of question writers who author hints for existing problems and/or compose their own questions with hints from the perspectives of the four Usable Math characters. Student writing offers important demonstrations of student learning. For example, to write a math story problem like the one about Mr. Man and his collection of cat pencil sharpeners, students need to know how to understand the concept and use a computational procedure to solve the problem they created. This math story demonstrates what the students know and are able to do with their current math knowledge and gives teachers ideas for additional instruction that will help the students further expand their thinking. In addition to designing math problems and solution choices, children also appear to have created the growth mindset statements. Writing takes time and students need materials, including paper and markers and/or digital tools so they can fully express their ideas. Deciding how to integrate math problem writing becomes another way for teachers to engage students in ongoing math problem solving.

Conclusions

From its debut in early 2023 with a module about fractions, Usable Math, a redesigned online math tutor for elementary-school age learners, teachers, and families, has expanded to 18 modules and more than 125 math word problems. Students in a 4th grade school classroom and 4th and 5th graders in a summer-long recreation program have used the system. We collected feedback about system design and function from them as well as from teachers in professional development workshops and students in college classes.

In developing the site, we encountered 5 decision points about how to design the system to best support student math learning. We have highlighted those decisions here. As we made our choices, we realized that teachers in classrooms face similar decisions as they seek to promote math learning and math problem solving by elementary students while addressing school system curriculum goals. Teachers may or may not arrive at the same conclusions that we did, but our experiences may serve as useful guides for their thinking.

Based on initial responses of users and our decisions as system authors and programmers, highlighted in this paper, we offer the following conclusions intended to guide educators to think about the ways to utilize digital materials for math learning in elementary school classes:

  • Students and teachers alike welcome opportunities to access online math learning resources, especially when these resources provide interactivity and choices as the four coaches offer with their individually crafted strategies of math problem solving.

  • Students are interested in engaging with the distinctive Usable Math coaches who serve as guides providing innovative ideas to aid math learning.

  • Students enjoy and appear motivated by seeing growth mindset affirmation statements after each problem that acknowledge their deliberate thinking and invite them to do more.

  • Four coaches with different problem solving approaches expand the thinking of children and adults, promoting the idea that everyone is engaging in learning multiple maths, not just one way to do math or to solve a certain type of word problem.

  • Having students write their own math questions, author math problem solving strategies for the existing coaches, and create entirely new characters who guide learning for math stories represents exciting new dimensions for integrating Usable Math in learning settings.

As we continue to expand the Usable Math site, we will be sharing more research about how students and teachers are using our online tutoring technology in classrooms to make learning math exciting parts of the mathematical content.