Keywords

1 Introduction

Visually impaired students have difficulties in perceiving graphical information. They normally use the senses of hearing and touch to obtain information from their surrounding environment. With the help of technological advancement, many techniques and solutions have been introduced that could assist them to improve their learning skills. Smartphone-based and computer-based solutions are prominent areas of solutions for visually impaired people and provide effective assistance to them in accessing information. In these solutions, artificial intelligence-based techniques are quite useful for visually impaired students. Shubham et al. [1] proposed a smart personal assistance by using the artificial intelligence technique that provided better access to the surrounding environment of visually impaired people. They used image recognization, text recognization and machine learning techniques together in form of a mobile application. This application helped visually impaired people in certain tasks i.e., accessing books electronically, currency and image recognization. Some tutoring applications are also available for visually impaired people that provide better ways to access and manipulate information during their learning process. The i-Math is an automatic math reader application which was designed for Thai Blind people and is quite useful for them to access mathematics material. It can read the mathematics content from documents in audio form with the help of a screen reader. Teachers also have the facility to prepare their assessments for students in the audio form [2]. Computer Haptics provided a new way to access information by touch, auditory and visual means. In this approach computer and haptic feedback controller were used by students to virtually explore the 3D shapes [3].

Interactive learning platforms are also becoming popular in education, especially for visually impaired students. Visually impaired students can easily use them in learning mathematics. These platforms used various technologies such as screen readers, braille displays, and audio instructions to present mathematical concepts in a way that was accessible to visually impaired students. One such platform is MathTrax [4], which was designed to offer accessible math materials to students with visual impairments. MathTrax used screen reader technology to read mathematical expressions aloud and allows students to navigate through equations using their keyboards. Another platform, MathSpeak [5], was a browser-based tool that can be used by both sighted and visually impaired students to learn math concepts through a combination of text, audio, and video instruction.

This paper introduced an Artificial Intelligence-Based Interactive Learning Platform for math tutoring of visually impaired children. This platform helped visually impaired children in learning mathematics by using text-to-speech and vibrotactile feedback. Visually impaired children can learn and practice linear and quadratic equations of mathematics by using this platform. The remainder of this article is organized as follows: Sect. 2 which defines previous work associated with visually impaired children learning mathematics. Section 3 describes the proposed methodology of the interactive learning platform. Section 4 presents the participants and results information. Lastly, Sect. 5 has a discussion and conclusion based on the results of this study.

2 Related Work

Ávila-Soto et al. designed a tangible user interface by using a multimodal method. This method used auditory feedback and tangible numbers to enhance the basic arithmetic skills of visually impaired students [6]. Bouck et al. proposed a computer-based Voice Input and Speech Output (VISO) calculator to help visually impaired students to solve basic mathematic calculations. Results showed that the VISO calculator reduced the time to perform the calculations. It also has some accuracy issues in the recognition of speech [7]. Bier et al. designed and implemented a multi-purpose math-to-speech translation system based on Lua script, which allows users to manually adjust translation rules, i.e., language change and focusing on the relevant requirements. Results illustrated that the overall success rate of the correct answers was about 83%. This system can be used for several purposes i.e., e-learning platforms for auditory representation and as an educational tool for non-native visually impaired users [8].

Buzzi et al. developed an accessible vibro-tactile-based android application to solve geometry problems. The design of this application is more accessible for visually impaired children because they can make freehand drawings and recognize geometry shapes based on vibration and touch senses [9]. Gulley et al. introduced an auditory approach to guide and evaluate mathematics learning named Process-Driven Math. It facilitates the visually impaired students who do not have enough experience in Braille to learn mathematics. Audio rendering of the algebraic expression was used to significantly reduce the complexity of mathematical expression. It also helped the visually impaired students to improve their understanding of mathematical expressions [10].

Murphy et al. designed auditory cues by using synthetic speech, non-speech and spearons. These newly designed cues were used to eliminate the ambiguity of mathematical formulas. They designed an online survey to evaluate the effectiveness of these auditory cues on 56 participants. Results demonstrated that visually impaired users better understand mathematical equations with the help of synthetic speech Their listing time is reduced with the help of modified speech (spearcons) [11]. MathPlayer is also an important platform for learning mathematics. It can read aloud mathematical formulas and has support for Internet Explorer, Mozilla Firefox and screen readers [12]. Furthermore, researchers have also proposed some theoretical methods for visually impaired students to learn specific kinds of mathematical problems i.e., algebra and calculus. Islam Elkabani and Rached Zantout proposed a framework that helped visually impaired people in learning linear algebra [13]. There is another framework that has been proposed to solve some simple calculus problems and achieve successful outcomes by Sebastian et al. [14].

3 Proposed Methodology

In this study, a prototype was introduced that can enhance the learning skills of visually impaired students more effectively. This prototype used a multimodal feedback approach that facilitates visually impaired students. This prototype has two main modules “interactive learning module” and “Practice module”. In the interactive learning module, students can practice linear and quadratic equations. They can choose one type of mathematical question, and the proposed system automatically searches the related questions from the question bank. They can interact with the system through their voice. During the interaction, the user can explore all relevant information about the selected questions. Shown below is a step-by-step example for solving a linear equation with one variable in this module:

Solve for y, y2 + 4(y + 1) = 1.

y2 + 4y + 4 = 1:

Multiply small parenthesis by 4

y2 + 4y + 4 – 1 = 0:

Move the right constant to the left side

y2 + 4y + 3 = 0:

Subtract both constant values

y2 + 3y + y + 3 = 0:

Factor the equation

y (y + 3) + y + 3 = 0:

Take y value common from equation

(y + 3)(y + 1):

Splitting the equation for the solution

y =  − 3, y =  − 1:

Final solution

Fig. 1.
figure 1

Provides a basic overview of the proposed framework.

Full size image

Once students have a proper understanding of the selected topic then they can proceed with the practice module to check their learning skills. Students can attempt to solve the questions in this module. They can acquire the information by using multimodal feedback which is the composition of text-to-speech and vibrotactile feedback. They have to select a question from the question bank. After selection, the text-to-speech function reads the selected question aloud which indicates the user is ready to solve the question. Users can give the step-by-step solution by speaking and the system can validate the solution. If there is something wrong then it generates vibrotactile feedback. Vibrotactile feedback indicates that something went wrong and the user can request a hint in this situation. If the student solved the question accurately then it generates a message that “problem solved”. Figure 1 provides a complete overview of the proposed framework.

4 Participants and Results

Four participants 3 male and 1 female, aged between 12 and 14 years old (M = 12.5 and SD = 0.84), were involved in this study. All participants were desktop computer and smartphone users. Table 1 provides the demographic information of the participants. Self-assessed scores in the table expressed the expertise level of the participants with desktop computers and smartphones. On average self-assessed expertise level with desktop is 5.25 and smartphone is 6.75. Year of usage represents the number of years that a participant has been using a desktop and smartphone. On average, participants have been using a desktop for 3 years, and a smartphone for 5 years.

Table 1. Participants’ demographic information.
Full size table

The study's goal was clearly explained to the participants. The details of application usage were thoroughly explained to them. Participants were told that in order to participate in this study, they must first attend a training session. All participants were informed that the entire study would be recorded on video for analysis purposes. Signed informed consent forms were also collected by participants.

Each participant responded to a survey after the experiment. The scores for this survey were reported as feeling pleasant (3)-unpleasant (1), like (4)-Dislike (0), enjoyable (4)-hurting (0), happy (4)-painful (0), and pleased (3)-irritating (1).

Fig. 2.
figure 2

Extracted word clouds from the comments of our participants (Designed from: https://www.wordclouds.com)

Full size image

Figure 2 displayed word clouds made from a total of 24 words that our participants believed best captured their experience with our proposed application. Mostly the words were positive i.e., “helpful”, “pleasant,” “enjoyable” “useful,” and “interesting”. As well as there were some words with negative responses i.e., “disturbing,” “painful” “useless” and “irritation” (Fig. 2).

Fig. 3.
figure 3

Interpretation of participants behaviour after the experiments

Full size image

After the experiments, we conducted a study with participants in which they had to fill out a questionnaire. We designed this questionnaire using the 5-Point Likert Scale (values as Strongly Agree, Agree, Neutral, Disagree, Strongly disagree). Questions were designed to measure several aspects of the participants i.e., satisfaction, action awareness, ease, skills improvement, understanding, helpful,interesting and enjoyable. Figure 3 provided an overview of the participants’ behaviour after the experiments. Blue indicates that participants strongly agree, maroon indicates that they agree, cyan color indicates that they were neutral, light purple indicates that they disagree and dark purple indicates that they strongly disagree with this study. It was clearly illustrated that participants mostly agreed or strongly agreed with the proposed prototype.

5 Discussion and Conclusion

Teaching mathematics to visually impaired students is a very difficult task due to the rich visual information i.e., graphs, formulas and notations. With the help of advanced technology, visually impaired students can now access a more inclusive and personalized learning experience that caters to their requirements and adjusts to their learning styles. Previous studies showed that auditory and vibrotactile feedbacks were very useful for visually impaired students in learning mathematics. In this study, we have combined both feedbacks to provide better interaction and action awareness to visually impaired students. Our main focus was to solve linear and quadratic equations of one variable. The proposed prototype was mainly designed for visually impaired students who want to gain better mathematics skills.

Our proposed prototype plays an important role in making mathematics material accessible to visually impaired students. With the help of auditory and vibrotactile feedback, visually impaired students can receive audio descriptions of math problems, interact with the platform using voice commands, and access visual representations of math concepts through an audio form. This prototype can track the student's progress and provides personalized feedback, enabling them to learn at their own pace and adapt to their individual learning styles. Results from the questionnaire data and face-to-face interviews showed that the proposed prototype helped visually impaired students to improve their learning skills in mathematics. Visually impaired students feel satisfied with multimodal feedback. Words clouds showed that visually impaired students felt enjoyable, pleasant and happy with the proposed prototype. 5 Point Likert Scale data represented that mostly participants agreed on better action awareness and understanding of the proposed prototype.

This prototype has the potential to significantly improve the accessibility of math education for visually impaired students. It was also effective in helping students to learn and understand mathematical concepts. However, further research is needed to introduce future improvements. Such as, we have recruited only four participants. In future, we can recruit more participants to better assess the application's usability and effectiveness to add a future enhancement. The final release of this application will be open-source and available for anyone to use for free to learn mathematical skills. We are also planning to link a parent and teacher account with the student, therefore, they can easily monitor the learning activities of the student.