Using Augmented Reality Interactive System to Support Digital Electronics Learning

Abstract

The fundamental theories and concepts of digital electronics which usually being taught at entry level electronics courses is designed to help the students develop solid underlying knowledge of how computer works. The topics usually covered Boolean algebra, Logic Gates and etc. Beside Theory, which are being taught in the lectures, practical skill is required by doing the experiments. In the past the digital electronics laboratory is needed for this purpose, but recently, it was replaced by the software that can simulate the digital circuits. The advantage of this software is that, the students can study the behavior of a systems without building them which save a lot of time and expense. The drawback of these simulators is that, it lacks real charisma in performing the experiments and difficult to interpret. In this research, we have applied interactive marker-based augmented reality (AR) to solve these problems. The advantage of using this marker based techniques in this research is that we can implant the functions of each logic gates in the marker and program these markers to interact with others just like in the real logic gates laboratory. The other advantage is that, markers can be in any shape or have any pictures or symbol on it which make user recognize them easily. In this research, we have created 9 markers which represent 7 basic Logic gates i.e. AND gate, OR gate, NOT gate, NAND gate, NOR gate, XOR gate and XNOR gate. Since we can implant the functions of each logic gates in the marker, the output from these markers will represent the function of each logic gate. By adding them together, the output will be the same as adding the logic gates together. The advantage of using this marker based AR techniques is that we can construct the digital circuit using these markers and the output will be just the same as using the real logic gates, except there is no need for power supply and can be done anywhere and anytime.

1 Introduction

Most students who took the courses related to computer or electronics must understand the basic of digital system. Digital systems are constructed by using logic gates such as AND, OR, NOT, NAND, NOR, EXOR and EXNOR gates. The basic operations can be described by using the truth table, which is a logically-based mathematical table that shows all the possible outcomes of the scenario. Beside theory, usually practical work is required by doing the experiments. In the past the digital electronics laboratory is required and all the equipment such as IC chips, breadboards, IC Testers, power supplies and etc. should be provided in the laboratory. Even though, the cost for setting up the lab is not so high but there must be at least one technician to take care of the setup. Recently, there are some software that can simulate the digital circuits, so they can replace the real hardware experiments. The advantage of these software are that, the students can study the behavior of the systems without building them. Besides that, the results are quite accurate compared to analytic model and it is also easy to perform “what if” analysis. The drawback of these simulators, beside the cost, is that, it lacks real charisma in performing the experiments and sometimes it is difficult to interpret the simulation results.

In this research, we have applied interactive augmented reality to solve these problems. Augmented Reality or AR is the technology that integrated the computer’s generated information or object into the user’s environment in the real world. Unlike virtual reality which creates the whole virtual environment, AR uses real environment and overlay virtual environment on top of it. There are two types of AR, i.e. marker based AR and marker-less AR. In marker based AR, computer will generated the virtual environment related to the marker but in marker-less AR, the generated environment that the computer generated are related to information provided such as GPS location. The advantage of using this marker based technique in this research is that, we can implant the functions of each logic gate in the marker and program each marker to interact with others easily just like in the real logic gates. The other advantage is that is marker can be in any shape or have any picture or logo on it which make user recognize it easily.

2 Background

2.1 Overview of Augmented Reality

Augmented Reality (AR) is the integration of computer generated objects and the real world environment, in which the virtual object is overlaid on top of existing space. The computer generated objects that were projected on the real surface can be in any forms such as 2D image, 3D model, text, video or sound. Unlike virtual reality (VR) in which users are immerged into the computer generated virtual environment, AR let users experience virtual environment in their real space. To make it simple, in VR, users can see only virtual environment while in AR, users can see both virtual and real environment at the same time.

There are two types of AR application in terms of development i.e. marker based AR and marker-less AR. In marker-less AR, the coordinates or locations of the virtual objects were assigned by GPS, while in marker based AR, the position of the virtual objects placed on the real environment come from the marker. There are many type of 2D graphics that can be used as a markers in marker based AR such as, templates marker, 2D barcode marker, imperceptible markers, frame marker and etc. Template marker is black and white marker that has a simple image inside a black border. 2D barcode markers are markers that consist of black and white data cells. Imperceptible markers or image marker is the marker that uses natural color images as marker. Frame marker is the marker developed by Qualcomm that are transparent with the black border and black-white marker. Summarize of these markers are shown in Fig. 1. Today, 3D marker is also available.

Fig. 1.

2D markers; (1) templates (2) 2D barcode (3) Imperceptible (4) Frame marker

2.2 Logic Gate and Its Truth Table

A logic gate is a basic building block of a digital circuit, which performs a logical operation on one or more binary inputs, and produces a single binary output. Most logic gates, except NOT gate, have two inputs and one output. The status of the terminal at one time is either “0” or “1”. The status of the terminal will change as the circuit process the data. There are seven basic logic gates: AND, OR, XOR, NOT, NAND, NOR, and XNOR. The AND gate, is the logic gate that acts as the logical “and” operator, which mean that the output will be “1” or “high” only when both input terminals are “1”, otherwise it is “0” or “low”. When the output is “0”, it is called “false” and when it is “1”, then it is called “true”, that is why the name “truth table” comes from. The “truth table” is a table that lists all the truth value of the outcome for each of the possible combinations. The OR gate is the logic gate that behave as a logical inclusive “or”. The output is “true” if either or both of the inputs are “1”. The XOR (exclusive-OR) gate behaves like the logical “either/or.” The output is “true” if either, but not both, of the input is “true”. The NOT gate or inverter, which has only one input, has the function that reverses the logic state. The NAND gate operates as an AND gate followed by a NOT gate. In other word, the output from AND gate will be the input of the NOT gate and the output of the NOT gate will be the output of the NAND gate. For the NOR gate which is a combination OR gate followed by an inverter. Its output will be “true”, only if both inputs to the OR gate are “false”. The XNOR (exclusive-NOR) gate is a combination XOR gate followed by an inverter. Its output is “true” if the inputs to the XOR gate are the same. The gate symbols, Boolean Algebra and truth table of seven basic logic gates (NOT, AND, NAND, OR, NOR and XNOR) are shown in Fig. 2. By using combinations of these logic gates, complex operations can be performed.

Fig. 2.

Logic gate symbols, boolean algebra expression and its’ truth table

3 Related Work

Augmented Reality Technology has been applied in many areas ranging from manufacturing, architecture, arts, entertainment and education. Since the unique capabilities of augmented reality to display information in almost any form of media, the potential for learning process is very promising. Many researchers have studied about how to use AR in education. Chen et al. [1], review more than 55 papers related to applying Augmented Reality in education that published between 2011–2016 and mentioned about the trend, future and opportunities in using augmented reality for education. Boonbrahm et al. [2], investigate on how to use AR to motivate children in learning English. Different AR techniques were used for this purpose i.e. marker-marker interaction and user-defined target. The results agree with the prediction that children really enjoy and eager to learn more. Fan et al. [3] indicates the significance of AR based experiment in education and mentioned that this technology will bring lots of new features for experimental education. Aquirregoitia Martinez et al. [4] presented the results of using AR applications for preschool education. The area of applications covers reading, numbers and etc. The conclusion indicated that the applications encourage interaction, content adaptability, multisensory stimulus and can be successfully adopted for learning and development in Preschool education. In the area of using AR Interface for Laboratory Experiments, Onime et al. [5], presented an application of Augmented Reality (AR) technology for hands-on practical laboratory experiments for Science and Engineering students and confirmed that the students found the AR technology easy to use, and they were satisfied with the simulation and rendering quality of the implemented AR applications. Boonbrahm et al. [6], show that adding physical properties to the virtual object in AR experiment will make interaction between virtual objects feel more realistic. Srivastava et al. [7] use Augmented Reality based Instructions for Electronics Lab (ARIEL) and tested amongst undergraduate students of Basic Electronic Engineering Laboratory course. This paper discusses the methodologies utilized for the design process. Even though, there are many AR applications being used in many fields of education but very few are available for replacing the real laboratory experiment, so there still be the opportunity to research into this area.

4 System Development

The virtual Digital electronics system development can be broken down into two parts i.e. AR applications for mobile and system setup.

4.1 Application Development

In this research, Unity3D game engine was used on Qualcomm’s Vulforia platform. Unity is a fully integrated development engine for creating games and other interactives 3D content. Qualcomm’s Vulforia platform made it possible to write a single native application that run on almost all smartphone and tablets. This application is designed for iOS system but can be modified to be used on Android as well. For the marker, frame marker was selected for this purpose since it is easy to detect.

4.2 System Setup

In the real setup of digital electronics experiment, all the equipment must connected to each other via wires or cables, but for virtual experiment, connection via wire is impossible. The only connecting path between each virtual parts i.e. logic gate markers, input/output markers, can be done through vision only. By placing logic gate marker on spot, the applications program in the smartphone will scan for the marker on the left sides which is the input and get the data for processing by logic gate markers’ function and then send the output to the marker which is located on the right side (output marker). Since almost all the logic gate has two inputs, the program will scan for two markers and get the data from both inputs before transfer the data for processing. After processing, the output data will be processed by next logic gate’s markers or displayed as the output.

4.3 Virtual Configuration

Since the markers in marker-based AR can be programmed to represent the virtual objects in both location of the object and mathematical functions, then the interaction between markers is possible. For example, if we have three markers that represent alphabet “D”, “O” and “G” and put them in that consecutive order, the program will process and interpret them as a word “DOG” and with the AR application, the 3D model of the dog will be displayed on top of the marker as shown in Fig. 3. The same concept can be applied to the logic gate as well.

Fig. 3.

Marker-marker interaction

4.4 Mapping AR and Logic Gate

In order to make a virtual logic gate circuit work just like the real logic gate circuit, the function that it performs should follow the truth table of that logic gate. In real logic gate, the input of the logic gate should be connected to the input source and the output should be connected to the display device or to another input of the next logic gate. To duplicate this process for virtual setup, there will be at least four AR markers, two for the input which can be either “0” or “1”, one logic gate marker and one output marker. Each of the markers can have the picture or symbol that show their function. For example, markers that have the number “0” and “1” are the input source and will give the value either “0” and “1”. Marker that shows the symbol of the logic gate should perform as that logic gate. Marker that has the bulb symbol is the display marker and it will perform as a light bulb, i.e. it is dark if the signal is “0” and bright if the signal is “1”. Example of the virtual setup for different logic gate are shown in Figs. 4, 5 and 6.

Fig. 4.

Truth table, setup for AND gate and its operation

Fig. 5.

Truth table, setup for OR gate and its operation

Fig. 6.

Truth table, setup for NOT gate or inverter and its operation

5 Implementation

In real digital electronics circuit, there are many logic gates connected to each other, so in the virtual circuit, the virtual logic gate should be able to do the same thing. In order to make this AR logic gate replace the real one in the lab, these AR logic gates must be able to interact with each other, giving a larger circuits that is useful in construct parts of the computer. As mentioned before, the markers in marker-based AR can be programmed to represent the virtual objects with mathematical functions, then the interaction between markers is possible. To demonstrate the possibility, we have constructed larger circuits consisted of many logic gate as shown in Figs. 7, 8 and 9.

Fig. 7.

Output from AND gate and NOT gate were used as an input to the OR gate.

Fig. 8.

Output from OR gate and AND gate were used as an input to the NAND gate.

Fig. 9.

Output from OR gate and NOR gate were used as an input to the AND gate.

6 Summary and Conclusion

From this research, we have found that, virtual Logic gate can replace real Logic gate for the laboratory experiment. The advantage of using the virtual experiment is that, the experiment can be done anywhere, anytime without using using the facilities in the laboratory. The only device needed is the smartphone or tablet with AR application program and paper markers. The disadvantage is the limitation of the space that can be clearly detected by the mobile phone camera. The suitable area that the mobile phone camera can detect nicely is A4 paper with the distance between camera and markers is around 12 to 20 inches. This concept can be adapted to cover other areas of experiments such as basic electronics, Physics, Chemistry and etc.