1 Introduction

Smart devices have an important role in our daily life, and from last decade, the rapid growth of these devices have achieved the essential part in our daily routines. Generally, it is very common for a single user to use multiple smart devices. However, it is very difficult for such user to synchronize data among these devices that increase the productivity and usability of these devices. Many companies involve different kind of protocols in these devices in order to enable data synchronization; among them, Bluetooth is one of the common solutions to handle this issue. This technology exists in many products, such as telephones, tablets, media players, robotics systems, handheld, and laptops. Another communication technology exists in near field communication (NFC), Quick Response (QR) Code, and techniques based on gesture recognition.

All these pairing techniques have several unique advantages. Commonly, these authentication and pairing procedures are very awkward for normal users, which effectively declines the user experience and practicality of the smart devices. For example, the users require searching the target device among the lengthy lists of devices when pairing takes place in one to many and many to many communication protocols [1]. Moreover, device identifiers in these lists are ambiguous for many users. Furthermore, to pair user phone with target device, user needs a mechanism to let user’s phone know which nearby phone is the intended target. This involves bridging a “perception gap” in device pairing [2].

NFC offers an easy way for paring by simply requiring a close proximity between the devices to be paired. However, in the NFC, the pairing is less transparent and man-in-the-middle attacks cannot be prevented which is one of the main limitations of the NFC. Few approaches require additional hardware, which make the overall design bulky. In these approaches, there are chances of errors when the user behaves naturally. Some methodologies require that the target device must be in sight of the user, and the pairing is affected if there are any obstacles between the user and the target device. Moreover, when other people surround a user, it makes the pairing difficult.

Fig. 1.
figure 1

Pairing between devices using the tangible tokens of the same frequency.

Full size image

There are several file sharing and data synchronization applications, which have received much praise and acceptance because of the simplicity of pairing; however, most of them are still inconvenient to use and lack the tangible benefits of physical interaction. Therefore, we have presented a new method of authentication and pairing smart devices based on physical and tangible tokens. By utilizing physical tokens, our proposed method is easy to adapt and intuitive because it only requires a simple touch with a tangible token to initiate the pairing between the two devices. Figure 1 shows a conceptual diagram of the proposed method. A token (that has its own unique ID) is activated using a button. To initiate a pairing between two devices, a tangible token has been placed on the surface of a smart device due to which it starts vibrating. By detecting this vibration pattern, the system is capable to determine the token ID. Then, by placing the same token on a second device, a connection through WiFi is established between these two devices, which share the same token ID. The proposed tangible system is composed of unique tokens that are placed on smart device. The smart device identified these tokens based on vibration frequency. We used this technique for the initialization of the pairing session between multiple devices.

The rest of the paper is organized as follows. Section 2 presented detailed discussion on related works. The proposed prototype and its implementation are presented in Sect. 3. Section 4 describes the experimental setup for the proposed system. The experimental results and discussion are presented in Sect. 5. Finally, the paper is concluded with some future directions in Sect. 6.

2 Related Work

Utilizing the communication protocols such as Bluetooth and NFC, the applications provide easy and intuitive solutions for data sharing and synchronization. Bump and S-beam are the best examples of such applications that enable the smartphone users to transfer contact information, photos, and files between devices. To initiate a transfer in bump, two people physically knock their phones together. Similarly, S-Beam application requires the smart phones to touch each other back to back in order to initiate the pairing and data transfer. There are also many camera-based solutions for household device selection. QR Code is the case for integrated cameras that can be used to read visual codes. PresiShare [3] presented new interaction technique that leverages the affordances of QR codes to accelerate device pairing and content sharing.

Gesture recognition has been widely studied which provides a device pairing. Point&Connect [2] identified the target devices by the user physically pointing the target. The system captured the user’s gesture, understood the target selection intention, and completed the device pairing. Moreover, this research suggested to further minimize this perception gap with a new intention-based device pairing paradigm. The system captured the intention of device selection via a simple pointing action. Similarly, Toss-it [4] allowed a user to identify targets and sent information to device in intuitive manner by utilizing their mobility when users perform a “toss” or “swing” actions in the intended direction, which were by the user in the real world. Air-Link [5] and DopLink [6] described air gesture recognition-based method, which detected the direction of movements using Doppler effects. Further, it allowed users to share files between multiple devices using in-air gestures by waving the hand from one device to another. In addition, users can easily exchange information such as photos between multiple devices.

Several existing approaches have been developed that have traditional input modality. TUI attempts to bring physical controls back into the reality. This combination of touch and TUI brought user applicability, feedback, and intuitive recognition [11]. Chan et al. in [7] presented Capstone, which used tangible blocks that allowed the underlying capacitive touchscreen to identify blocks and to sense how they arranged in 3D. The presented tangibles are designed to provide additional functionality to tangible-enhanced applications such as tangible board games or applications with tangible controls. Kratz et al. in [8] created artificial touch points using tangible object for creation of a multitude of physical controls. Their technique allowed the creation of a multitude of physical controls for capacitive touch screens. Ranging, for instance, getting rotary knobs or sliders from simple styli. Bianchi et al. in [9] presented tangible interfaces based on the magnetic sensing. This technology enabled various interfaces with mobile devices by exploring the design space of embedding magnets in tangible tokens. This technology has challenges in sensing the magnetic fields to create range of physical prototypes of interactive objects.

3 Prototype

3.1 Hardware Design

In this section, we have presented a hardware design, which is used to control the vibration frequency for identifying a single object. This prototype of a tangible token consists of a small functional Arduino (Gemma), coin shape linear vibration motor, battery, and button (as shown in Fig. 2a). The resultant shape of the proposed prototype is 37 × 37 mm cylindricality shape that is 3D Printed (as shown in Fig. 2c). The Gemma is located inside the 3D shape, which is a microcontroller board based on the ATtiny85. It has three digital input/output pins, micro USB connection, and JST connector for a 3.7 V battery. Moreover, we have placed a vibration motor that has 2-3.6 V operating range in the bottom. 3.7 V 280 mAh polymer lithium ion battery is used to keep the device small. To facilitate the detection of the touch input on screen, we positioned the vibration motor and touch material in the bottom side (as shown in Fig. 2b). It enables to detect the touch input in a precise manner. A 3.7 V battery is placed over the vibration motor that is attached to Gemma. Gemma enables the user to manipulate the vibration frequency by pressing the button on the top. The proposed prototype is designed to distinguish three different frequencies (2 Hz, 4 Hz and 6 Hz), and when the user presses the button, then each frequency has been activated accordingly.

Fig. 2.
figure 2

Prototype (Vibration Token). (a) Circuit board and Actuator, (b-c) Case and Dimensions

Full size image

3.2 Software Design

We have developed an application that easily enables two devices to pair using tangible tokens based on touch interface and vibration frequencies. The proposed application consists of three modules such as touch manager, sensing the vibration analysis manager, and pairing manager. When the application is running, a vibrating tangible token with a certain frequency is placed on the screen. The smart device detects the token using the device touch event features, which are usually provided by the operating system. Using the touch event as a trigger, the smart device retrieves the raw data from the device accelerometers for a predetermined period (e.g. one sec). The vibration frequency of the token is detected using the raw data that was retrieved from the accelerometers in the mobile device. The changes in vibration frequency are also embedded in the raw data. We respective apply high pass and low pass filters on raw data in order to reduce the noise, and then convert the signal from its original form to a representative frequency form by using the fast Fourier transform (FFT) algorithm. After the filtering and conversion, current frequency of the object is determined. When touch event happens again, it automatically begins to collect the data for the next interval.

After determining the token’s vibration frequency, pairing manager is initiated to establish a connection with the other device with the same token frequency. The proposed application uses the open sound control (OSC) protocol to broadcast the data to other devices [10]. Based on OSC protocol, each broadcast packet contains the tokens ID (vibration Hz) and devices’ ID. When the other device receives the packet, it checks the devices’ ID for the device identification and then checks the token ID to match with its own devices’ token ID. We used OSC protocol because it does not require pre-authentication like other network protocols (Bluetooth, Wi-Fi Direct etc.). As the OSC protocol is weak on security and has some flaws, but we simply used OSC protocol to test the connectivity and send/receive function in the proposed prototype. Figure 3 shows the matching process of token ID for the pairing. Figure 3a shows initialization of single device using one token ID (Freq 2), Fig. 3b shows two devices initialized with same token ID (Freq 2), and hence, successfully paired. Figure 3c shows the case where devices are initialized with two different token IDs (Freq 2 and 6, respectively) that however, fail to make a pair because of different token IDs.

Fig. 3.
figure 3

Application. (a) Frequency detection on a device through token, (b) same frequencies enabling the devices to pair, (c) different frequencies preventing the devices to pair.

Full size image

4 Experimental Setup

4.1 Procedure

For a thorough validation and testing, we have conducted some experiments in order to understand the basic usability characteristics and accuracy of the proposed prototype. We recruited three undergraduate and seven graduate students (total ten people, three female and seven male) and all are right handed. The age range for all the participants are between 21 and 36 years (whose average is 26.1 with standard deviation is 4.04). At the beginning, a brief introduction about the proposed approach is delivered to the participants, and all of them are required to provide their personal information. In introduction, we described and demonstrated three different vibration frequencies, tokens’ interface, and sensing progress. Based on the instructions, the participants operate vibration token and conduct a pairing with each one of the vibration frequencies between 2–3 min, which was the process of the training.

For the testing (as shown in Fig. 4a), we have placed 4 devices on the table and all were connected to the same Wi-Fi network. The front three devices already set with different token vibration frequencies. In front of those devices, we put the paper showing the vibration frequency designated to a particular device. We separated the experiments in two tasks. We put one device in front of a participant, which can be either a Nexus or Galaxy S6. In the first task, all the participants perform the pairing set 6 times using the Nexus 5. Each set is organized into three trials; therefore, a participant in one task performs 18 pairing trials. After a short break time, in the second round, all the participants repeat the same process using the Galaxy S6. We randomly provided each pairing trial frequency to the user. Finally, all the participants filled a questionnaire and provided their feedback in a short interview. Each participant took approximately 20 min to complete all the tasks (such as brief introduction, training, testing, and short interview).

Fig. 4.
figure 4

(a) shows that three target devices are located with its vibration and a device is located in front of the user. (b) displayed that participant performs the trials in the task.

Full size image

4.2 User Studies

The performance of the proposed approach has been evaluated in order to determine the usability and effectiveness of the proposed pairing method, and observes the intuitiveness of the user. For this purpose, we have conducted some experiments for the analysis of the time taken by each participant while performing each trial and its error ratio. In addition, we measure the users’ recognition of different vibration frequencies. More specifically, we wanted to determine the tokens’ interface that effectively used in pairing method. We examine the recognition accuracy of vibration interface in the pairing process.

For this experiment, we recorded both the classified results and the actual command that was given to the participants. To do so, we recorded the software/human error and pairing times. The software error is counted when the vibration sensing is not correct. When user recognition of vibration is wrong or click the button by mistake, a human error is counted. The pairing time is defined as a period, which begins with certain frequency that is being told to the participant, and ends with the completion of device pairing. We processed the outcome of the approach by conducting a related process on user feedback, which focused on identifying the subset of tasks and activities presented in the scenarios that users reported to be important, interesting, and useful. The total number of trials analyzed was 360 (10 users × 2 tasks × 3 vibration frequencies × 6 sets).

5 Result and Discussion

Our experiments tested the users’ recognition of the vibration frequency and usability of the token interface. Figure 5a presents that average pairing time of each trial for a particular frequency when the pairing is successful. Figure 5b shows the percentage of the system error and Fig. 5c is about the percentage of the human error. As mentioned earlier, system error occurs when a token is on proper vibration frequency, but application fails to detect it properly. By human error, we meant that when the user activates the touch event with wrong vibration frequency.

Fig. 5.
figure 5

The average classification from evaluated data set. (a) presents pairing time of each vibration frequencies (i.e., 2, 4, 6 Hz), (b) shows system error of configuration of frequency; while (c) describes the average user error.

Full size image

Task 1 took average time 7.3 s with standard deviation (SD) 2.8. One-way ANOVA revealed no differences across frequency patterns. The 2 Hz pattern is the faster one having average time 5.8 s with SD = 0.9 followed by the 6 Hz pattern (7.6 s with SD = 3.3), and the 4 Hz pattern (8.6 s with SD = 3.2). Figure 5b shows the systems miss-classification; while Fig. 5c shows the number of human-made errors (miss-clicks).

Task 2 took average time 6.7 s with SD = 2.5 for completion, and one-way ANOVA revealed differences across patterns (F (27, 2) = 3.45, p < 0.05) with having again the 2 Hz pattern faster 5.2 s with SD = 0.8 followed by the 4 Hz pattern (7.4 s with SD = 2.2), and the 6 Hz pattern (7.7 s with SD = 3.3). It can be seen from Fig. 5c that errors were not statistically different.

These experimental results indicate that the 2 Hz pattern was the most usable, I.e., it is the fastest one that has least error. The 4 Hz and 6 Hz patterns led to very similar results and no difference among the two were found. However, based on the observation the users found that the 6 Hz pattern is easier than of the 4 Hz pattern. This is because, the users had to place the vibrating token on the screen after selecting the correct pattern using the button placed on the token. Overall users found that the 2 Hz and 6 Hz pattern are easier to recognize than of the 4 Hz one.

During the experiments, we found that the proposed prototype had some limitations; for example, it was difficult for users to change between the frequencies using the toggle button on the top. Nevertheless, the overall users’ performance with the prototype suggests that this type of interface was easy to understand and use and that pairing by using physical objects can potentially lead to faster and more intuitive interactions than the GUI interfaces. Nevertheless, the results are encouraging and this suggests that the proposed approach is more robust in dynamic environments.

6 Conclusion and Future Work

In this paper, we proposed a novel method for pairing multiple devices using the physical objects. The proposed technique uses different vibration frequencies to identify and differentiate between different users. The experimental results and user studies suggest that the proposed methodology can be feasible in any environment. Moreover, the combination of tangible token and pairing method contributes to expand the interaction that addresses the perception gap problem by exploring how the sensing capabilities can be leveraged to create the tangible interfaces. In the proposed system, we used the physical token that provides vibration frequencies. Smart device can detect specific vibration frequency when a token is on the surface of it by the developed sensing algorithm and the embedded sensor. This information is used in the matching process of token ID for the pairing. This procedure makes the pairing steps easy, i.e., the users do not need to memorize the device name and ID. Additionally, the proposed system reduces the process of authentication and target selection. We improved the pairing method, which allows user to interact more flexibly with tangible objects.

In the future, we will perform more experiments with diverse evaluation criteria. In addition, we will explore procedures for manual calibration and mechanisms to limit the influence of the environment. Moreover, we will plan to develop a prototype, which will be much smaller and manageable to use by improving button interface and system. It will detect a wide range of frequencies more precisely and will test the user recognition that will enable to show the true potential of our proposed method.