1 Introduction

After the decision for the actuator (Chap. 9) used to generate the haptic feedback, and after the measurement of forces (Sect. 10.5) or positions (Sect. 10.7), it becomes necessary to focus on the IT-interface. This interface has to be capable of providing data to the actuation unit and catch and transmit all data from the sensors. Its requirements result—such as with any interface—from the amplitude resolution of the information and the speed at which they have to be transmitted. The focus of this chapter lies on the speed of transmission, as this aspect is the most relevant bottleneck when designing haptic devices. Haptic applications are frequently located on the borderline, may it be with regards on the delay acceptable in the transmission, or the maximum data rate in the sense of a border frequency.

With regards to the interface two typical situations may be distinguished: Spatially distributed tactile displays with a reasonable number of actuators; and primarily kinaesthetic systems with a smaller number of actuators. In case of tactile systems, pin-arrays, vibrators, or tactors the challenge is given by the application of bus-systems for the reduction of cable lengths, and the decentralization of control. Although there are still some questions about timing left, for example to provide tactile signals in the right order despite of a decentralized control, the data rates transmitted are usually not a challenge for common bus systems. Van Erp points out [20], that a 30 ms time delay between impulses generated by two vibrators at the limbs may not be distinguished any more. For the data interface this observation implies for this application, that any time delay below 30 ms may be uncritical for transmitting information haptically. This is a requirement, which can be fulfilled by serial automation technology network protocols like CAN, or the time triggered version TTCAN, without any problems. Accordingly this section concentrates on requirements of haptic kinaesthetic devices with a small number of actuators only, whereas these devices usually have to satisfy tactile requirements according to their dynamic responses too (Fig. 11.1).

Fig. 11.1
figure 1

The components of haptics

2 Border Frequency of the Transmission Chain

Section 1.4.2 stated that it is necessary to distinguish two frequency areas when talking about haptic systems. The lower frequency range up to \(\approx \)30 Hz includes a bidirectional information flow, whereas the high frequency area >30 Hz transmits information only unidirectional from the technical system to the user. Although the user himself influences the quality of this transmission by altering the mechanical coupling, this change itself happens at lower frequencies only, and is—from the perspective of bandwidth—not relevant for the transmission. If this knowledge is applied to the typical structures of haptic devices from Chap. 6, some fascinating results can be found. For the following analysis it is assumed, that the transmission and signal conditioning of information happens digital. According to Nyquist, the maximum signal frequency has to be sampled at least two times faster. In practical application this factor two is a purely theoretical concept, and it is strongly recommend to sample and analog system around 10 times faster than its maximum frequency. The values within figures and texts are based on this assumption.

2.1 Bandwidth in a Telemanipulation System

For a telemanipulation system, (Fig. 11.2) the knowledge about the differing asymmetric dynamics during interaction gives the opportunity to benefit directly for the technical design. In theory it is possible to transmit the haptic information measured at the object within the bandwidth of 1 Hz to 10 kHz, and replay it as forces or positions to the user. The user’s reactions may in this case be measured at a bandwidth from static to 5 or 15 Hz only, and be transmitted via controller and manipulator to the object. Although this approach would be functional indeed, the simplicity of position measurement and the necessity to process them for e.g. passivity control result in movements being sampled and transmitted similar dynamic as in the opposite transmission direction for haptic feedback.

Fig. 11.2
figure 2

Block diagram of a telemanipulator with haptic feedback

2.2 Cloud-Enabled Communication

Cloud system is one of the ways for sending and receiving data remotely. There are some research works that have already provided their ways for remote communication. Ongvisatepaiboon et al. [17] proposes a remote communication between therapist and patient through a web server. A web server is a hardware device like a computer that connects to the internet and supports data interchange between devices. The cost of setting up a web server is high and also needs a cumbersome process on both sides while setting up the therapy. Cloud robotics being the main aspect of the Internet of Robotics thing (IoRT) has been emerging a lot nowadays [6]. Cloud servers are internet servers that don’t require any hardware. There are various reasons which can affect the time-lag in cloud communication between two robots. They are the quality of the internet service, speed of the internet connection, load on the system, and the distance of the device from the server.

Some papers used ROSLink [11] for remote communication that requires entering the I.P address of the device each time to connect and make it less user-friendly. Another way is to trust the cloud servers that has been utilized. This server can be effectively configured (much less effect on system load), it is easily reachable, has accurate data management, and high data privacy. The most important thing to consider is the connection between two robots which is possible without any user input.

In certain applications, cloud-based multi-agent systems have been enabled to share much information between two systems  [9, 14] using cloud-based systems. A similar type of approach could be utilized by using the libraries of cloud-based pub/sub provided by google server [7]. Data would be transferred by one robot becoming a publisher to another robot acting as a subscriber.

Bringing down the publishing speed to 35 Hz decreases the system load up to a larger extent. Only the important data of change in state is transferred from one robot to another. The generalized view of the robots connected to the cloud server by using ROS is shown in Fig. 11.3. Following the steps, various google topics were made for all the control parameters like position, velocity, current, PID values. You can look at the literature for more details on forming a google publisher and subscriber [8].

Fig. 11.3
figure 3

Connection of cloud system between two devices

As stated, the user-friendliness of the system is important. Upon powering the raspberry pi and giving the internet connection, all the algorithms and code of the system and google cloud will be activated, and robots would be ready for therapy. No user inputs or manual setup will be required. Figure 11.4 will show the designed start-up service in brief.

Fig. 11.4
figure 4

Schematic view of a start-up service

2.3 Bandwidth in a Simulator-System

For a simulation-system with haptic feedback the different dynamics results in slightly different findings. Nevertheless it is still true, that the movement information may be sampled at a lower rate. However the simulator (Fig. 11.5) has to provide the force output at a frequency of 1 to 10 kHz. Due to this simple reason, the simulator has to be aware of the actual position data for every simulation step. Consequently with simulators the haptic output and the measurement of user reaction has to happen at high frequency (exceptions, see Sect. 11.3).

There are two approaches to integrate the haptic controller in the simulator. In many devices it is designed as an external hardware component (Fig. 11.5), which reduces the computing load for the main simulator, and helps reducing the data rate significantly in special data processing concepts with parametrizable models (Sect. 11.3). As an alternative the controller may be realized in software as a driver computed by the simulation main computing unit (Fig. 11.5). This is a concept used especially for high power permanently installed simulation machines, or which is used in cost-effective haptic devices for gaming industry with little requirements in dynamics and haptic output.

Fig. 11.5
figure 5

Block diagram of a simulator with haptic feedback and an external controller

Fig. 11.6
figure 6

Block-diagram of a simulator with haptic feedback and a controller as part of the driver software

2.4 Data Rates and Latencies

Table 11.1 summarizes the data rates necessary for kinesthetic applications in some typical examples. The data rates range from 200 kbit/s for simple applications up to 50 Mbit/s for more complex systems. Such rates for the information payload—still excluding the overhead necessary for the protocol and the device control—are achieved by several standard interface types today (Fig. 11.6).

Table 11.1 Example calculating the required unidirectional data rates for typical haptic devices

Beside the requirements for the data rate there is another requirement considering the smallest possible latency. Especially interfaces using packets for transmission, with an uncertainty about the exact time of the transmission (e.g. USB) have to be analyzed critically concerning this effect. Variable latencies between several packets are a problem in any case. If there are constant latencies the reference to other senses with their transmission channel becomes important: A collision is not allowed to happen significantly earlier or later haptically than e.g. visually or acoustically. The range possible for latency is largely dependent on the way to present the other sensual impressions. This interdependencies are subject to current research and are analyzed e.g. by the group around Buss at the Technische Universität München.

2.5 What is a Raspberry Pi?

The Raspberry Pi consists of a series of single-board computers. It is small and low cost and can be plugged into a computer monitor or TV. Also, you can connect it to a standard keyboard and mouse. Furthermore, it provides the ability to run different languages codes such as Scratch and Python. It’s capable of doing everything that a desktop computer can do, such as browsing the internet and playing high-definition video, to making text files, and playing games.

Moreover, the Raspberry Pi can interact with the outside world. It has been used in many digital maker projects, from music machines to weather stations and infra-red cameras communication Fig. 11.7. So it makes this ability for a haptic system to have a communication with user and it receives the data from sensor and sends the commands to motor and by this way, it can provide tactile feedback.

Fig. 11.7
figure 7

Raspberry Pi 4 B

3 Concepts for Bandwidth Reduction

Whoever ever tried to process a continuous data flow of several megabit with a PC, and in parallel make this PC do some other tasks too, will have noticed that the management of the data flow binds immense computing power. With this problem in mind and as a result from the question about telemanipulation with remotely located systems several solutions for bandwidth reduction of haptic data transmission have been found.

3.1 Analysis of the Required Dynamics

The conscious analysis of the dynamics of the situation at hand should be ahead of every method to reduce bandwidth. The limiting cases to be analyzed are given by the initial contact or collision with the objects. If the objects are soft, the border frequencies are in the range <100 Hz. If there are stiff objects part of interaction and if there is the wish to feed back these collisions too, the frequencies up to a border >1 kHz will have to be transmitted. Additionally it has to be considered that the user is limited concerning its own dynamics, or may even be further limited artificially. The DaVinci System (Fig. 1.12) as an unidirectional telemanipulator filters e.g. the high frequencies of the human movements to prevent a trembling of the surgical instruments.

3.1.1 Example

As an example, a varactor capacitance can be considered. It is a kind of semiconductor diode used in radio frequency tuning circuits. A Varactor Diode is a p-n junction diode and acts as a variable capacitor under a varying reverse bias voltage. It is a specially designed semiconductor diode whose, by modifying the applied voltage on its terminals, adjusts the capacitance at the p-n semiconductor junction Fig. 11.8.

Fig. 11.8
figure 8

Schematic view of a varactor structure [4]

Thus, the frequency control of a varactor device can be done by applying different voltages to the tuning port. It is united with a low impedance capacity and provides a high impedance to the driver. Varactor devices are more straightforward for having the same sweep speed driving than YIG devices, a ferrite with a sharp ferrimagnetic resonance and very high resistivity. These characteristics allow YIG resonator oscillators to have vast tuning ranges: 2–20 GHz tunable oscillators are available. Electromagnets are an important part of the YIG oscillators module, and frequency tuning of the YIG resonator is performed by modifying the currents in these electromagnets.

3.2 Ergonomic Standards for Haptic and Tactile Interactions

Ergonomics is the method of designing or arranging products, workplaces, and systems to suit more for people’s usage.

Most people consider ergonomics to do with seating or with the design of car instruments, but it is so much more. Ergonomics concerns the design of anything that includes people, from workspaces to leisure and safety. Ergonomic standards help improve usability in several ways, including improving effectiveness, reducing errors, increasing performance, and comfort. Ergonomic standards provide a base for analysis, design, evaluation, procurement.

One part of the standard series is a framework for tactile and haptic interaction. It provides a structure for understanding the different aspects of tactile/haptic interaction and communicating them. Different definitions, structures, and models used in other parts are included in this part. Moreover, it provides general information about how different forms of interaction can be applied to different tasks. Many efforts were made to define haptics terminologies  [16, 17].

3.2.1 Example

While there is no difference between haptic and tactile in most dictionary definitions [2], many researchers use tactile for skin mechanical stimulation and haptic for all haptic sensations. The framework document explains interaction details and task primitives for haptic interaction. Users can start application tasks using one or more task primitives enabled by the haptic device and its software. Task primitives are modified according to the system functionality.

Furthermore, the framework document presents guidelines for the ergonomic design of different haptic interactions, interaction space, convenience, and resolution. In addition, this part proposes haptics physiology, device types, haptics application areas, and selection criteria. Figure 11.1 shows the relationship of the different components that make the field of haptics.

The other part of the document is the guidance on tactile and haptic interaction. This standard contains guidance in different areas. The first one is applicability points for haptic interactions, including effectiveness, efficiency, Workload, user satisfaction, user needs, accessibility, security, health, and safety considerations. Another one is tactile/haptic inputs, outputs, and their combinations such as unimodal and multimodal interactions, individualization, and user perceptions. The properties of objects are categorized as tactile/haptic information attributes. The document should also contain the layout of the tactile/haptic objects such as resolution, consistency, and separation. Last but not least could be interaction data such as interaction tasks, navigation, manipulation, techniques, gesturing, and encoding by using textual data. For example, reading tactile alphabets guidance suitable for the blind is controlled by Unicode and national standards.

The last part of the standard is some measures to characterize haptic devices and user capabilities. The base of the design and evaluation of haptic/tactile interactions is characterizing physical properties. It includes the development of new devices and their requirements. The part mainly comprises the description of sets of measures and corresponding measurement setups, such as physical measures specifying technical characteristics of devices and human performance criteria related to perception, frequency, and operation speed.

In this regard, since 2005, an ISO expert group has been working on standards certificates for haptic interaction. ISO TC159/SC4/WG9 published its progress at some conferences  [1, 5, 19] and issued its first standard in 2009 [10]. Here there is a list of different standard series:

  • ISO 9241-900: Introduction to tactile and haptic interactions providing an overview of the 900 series. It also guides how various forms of interaction can be used for a variety of tasks. Making use of tactile/haptic systems, it is applicable to all types of interactive devices.

  • ISO 9241-910: Framework for tactile/haptic interactions, including a detailed list of terms and definitions. It also explains how a model can analyze, design, and evaluate interfaces with tactile/haptic interactions.

  • ISO 9241-920: Guidance on tactile and haptic interactions and ergonomics of human-system interaction. It gives recommendations for hardware and software interactions in tactile and haptic usages. It guides the design and evaluation of interaction hardware, software, and combinations of them.

  • ISO 9241-930: Haptic/tactile interactions in multimodal environments guiding specific to immersive and other multimodal environments.

  • ISO 9241-940: Evaluation of tactile/haptic interactions will guide evaluation methods for evaluating tactile and haptic interactions. This document applies to augmented reality, gesture control of a device or a virtual scenario, unidirectional interaction such as a vibrating phone or a vibrating belt, and virtual environment.

  • ISO 9241-971: Accessibility of tactile/haptic interactive systems. It provides general and specific ergonomic requirements and guidance for accessible tactile/haptic interactive systems, including accessible tactile/haptic interactions. In addition, it guides the convenience of interactive systems using tactile/haptic input/output modalities such as gestures, vibration, and force feedback. The guidance also supports alternative input modalities and different output representations usages.

3.3 Local Haptic Model in the Controller

A frequently used strategy being part of many haptic libraries is the usage of local haptic models. These models allow a much faster reaction on the user’s input compared to the simulation of a complete object interaction (Fig. 11.9). Such models are typically linearized functions dependent on one or more parameters. These parameters are actualized by the simulation at a lower frequency. For example each degree-of-freedom of the haptic system may be equipped with a model of spring, mass and damper, whose stiffness-, mass- and friction-coefficient is updated to the actual value at each simulation step, e.g. every \(\approx \frac{1}{30}\) s. This approach does not permit the simulation of nonlinear effects in this simple form. The most frequent nonlinear effect when interacting with virtual worlds is the lift-off of a tool from a surface. Dependent on the delay of the actualization of the local model, the lift-off will be perceived as “sticking”, as the tools is held to the simulated surface by the local model in one simulation step, whereas it is suddenly released within the next. Concepts, which model nonlinear stiffnesses, compensate this effect satisfactory. By making the additional calculations necessary for the local model, a significant data reduction between simulation and haptic controllers is achieved. Distantly related concepts are used in automotive applications too, where CAN bus-systems are configured in their haptic characteristics by a host, and report selection events in return only.

Fig. 11.9
figure 9

Block diagram of a simulator with haptic feedback and a local haptic model inside the controller

3.4 Event-Based Haptics

Kuchenbecker presented in 2005 the concept of “Event-based haptics” [12] and brought it into perfection since. It is based on the idea to split low frequency interaction and high-frequency unidirectional presentation, especially of tactile information (Fig. 11.10). These tactile events are stored in the controller and are activated by the simulation. They are combined with the low-frequency signal synthesized from the simulation, and are presented to the user as a sum. In an improved version, a monitoring of the coupling between haptic device and user is added, and the events’ intensities are scaled accordingly. The design generates impressively realistic collisions with comparably soft haptic devices. As any other highly dynamic system it nevertheless requires a specialized driver electronics and actuator selection to achieve full performance.

Fig. 11.10
figure 10

Block diagram of a simulator with haptic feedback and with events of high dynamic being held inside the controlling structure

A variant of the concept of event-based haptics is the overlay of measured high-frequency components on a low-frequency interaction. This concept can be found in the case of VerroTouch (Sect. 2.4.4) or in the application of an assistive system like HapCath (Sect. 14.2). The overall concept of all these systems follows Fig. 11.11. A highly dynamic sensor (piezoelectric or piezoresistive) is implemented in a coupled mechanical manipulation system. The interaction forces or vibrations induced by collisions between tool and object are then transmitted to an actuating unit attached near to the handle of the device. In case of these systems, it is then just a variant whether the interaction path is also decoupled or sticks to the normal mechanical connection.

Fig. 11.11
figure 11

Concept of an event based haptic overlay of tactile relevant data with (a) and without (b) mechanical coupling of interface and manipulator

3.5 Movement Extrapolation

Another very frequently used method for bandwidth reduction on the path to measure user reaction is given by extrapolation of the movement. Especially with simulators using local models it is often necessary to have some information about steps in between two complete measurement sets, as the duration of a single simulation step varies strongly, and the available computing power has to be used most efficiently. The extrapolation becomes a prediction with increased latency and a further reduced transfer rate. Prediction is used for haptic interaction with extreme dead times.

3.6 Compensation of Extreme Dead Times

The working group of Niemeyer from Telerobotics Lab. at the Stanford University works on the compensation of extreme dead-times of several seconds by prediction [15]. The dead-time affects both paths: the user’s reaction and the information to the user, such as the haptic feedback generated. The underlying principle is an extension of the telemanipulation system, which is added with a controller of the manipulator and a powerful controller for the haptic feedback (Fig. 11.12). Latter can be understood as an own simulator of the manipulated environment. During movement, a model of the environment is generated in parallel. If a collision happens in the real world, the collision is placed as a wall in the model, and its simulation provides a haptic feedback. Due to the time lag the collision does not happen at the position where it happened in reality. During the following simulation the collision point is relocated slowly within the model back to its correct position. By successive exploration of the environment a more detailed haptic model is generated. The method has the status of a research project.

Fig. 11.12
figure 12

Block diagram of a telemanipulator with compensation of long dead times by an adaptable world-model

3.7 Compression

As any data data stream, haptic data can be compressed for reducing their bandwidth. This may happen based on numerical methods on each individual packet, however it may also be possible to make use of the special properties of the haptic human-machine interaction and haptic perception. The following list shall give a short overview about common approaches:

  • A first approach for compressing haptic data is given in the situational adaption of digitalisation on the path for measuring user reaction. Shahabi [18] compares different digitalisation methods adapting their time- and amplitude-discretization on the actual movement velocity.

  • Since several years now the working group around Buss does intensive research on the perceptional impact of loss of resolution and bandwidth in haptic data streams [13]. They are coupling their research with the analysis of user-reactions, and base their algorithm on the psychophysical perception and an benefit-effort-analysis.

  • The working group around El Saddik wants to achieve data reduction by standardizing the haptic interaction in a descriptive data format “HAML”. It models the environment in a comparably little set of parameters, which gives advantages in tele-interaction applications with a larger group of participants. Especially varying data transmission paths can be compensated more easily on this abstract description, in comparison to classic telemanipulation approaches with a transmission of explicit forces and positions. As a by-product of this work, concepts for the unidirectional replay of haptic data in form of a “haptic player” are developed [3].

  • Another obvious approach for compression is the usage of limitations given by haptic perception. The working group of Kubica demonstrates [21] an analysis of an interaction with a virtual environment at different velocities. The identified dependency of the force perception threshold on the velocity was successfully used as basis for data reduction.

4 Specifications for a Portable Haptic Interface

Many haptic devices have never left the status of a prototype and grow into a commercial product. In most cases, the device design was the reason. One item that restricts the applications is heavy structure. Using electric motors, pneumatic cylinders, exoskeletons, and huge design dimensions are some of the reasons. The portable systems can make more freedom for users; however, nonportable devices take the massive weights away from the user. Involving a portable force-feedback interface is necessary to allow maximum motion freedom.

For a portable haptic system, different items are essential. As some general requirements:

  • simple design

  • short calibration

  • comfortability

  • low weight

  • low energy consumption

  • safety

  • low cost (manufacturing and parts)

A portable haptic system indicates an actuating and a sensing structure, and most of the time, it is attached to the user’s body. Because of different limitations such as overall weight and volume, portable devices are more challenging to design. On the other hand, its cable connections to different parts and the power supply are other challenging parts that should be considered. In these systems, low energy consumption should be considered because of the battery use. Easy usage, simple fitting structure, and short training periods are some other important specifications. Last but not least is electrical and mechanical safety. Electrical safety means that the device specifications must obey the international electrical safety limits. Also, mechanical safety should be in a way that prevents users’ injury. For example, there is no restriction in terms of mechanical force, but it is recommended to use the actuators with lower providing force than human power. Moreover, defining a suitable port fpr the system has an impact on user friendliness of the system.

FireWire—IEEE 1394

FireWire, Apple’s brand name, according to the IEEE 1394 standard is a serial transmission format similar to USB. In fact it is a lot older than the USB  specification. The six-pole FireWire Connector includes a ground and a supply line too. The voltage is not controlled and may take any value between 8 to 33 V. FireWire 400 defines up to 48 W power to be transmitted. The data rates are—dependent on the port design—100, 200, 400 or 1600 kbit/s. This is completely sufficient for any haptic application. Even fiber optics-transmission over 100 m distance with up to 3200 kbit/s are specified in the standard. The bus-hardware additionally includes a concept to share memory areas between host and client, enabling very latency less transmissions. Even networks without an explicit host can be established. The interface according to IEEE 1394 is the preferred design for applications with high data transmission rates. Only the little propagation of this interface in personal computers hinders a wide application.

Ethernet

The capabilities of the Ethernet-interface available with any PC are enormous. If you use a standard Ethernet network, you have the ability to transmit data at a rate up to 10 Megabits per second (10 Mbps). The available data payload within the transmission is largely dependent on the interlacing of the underlying protocols. A reliable 2-way data stream between remote applications is provided by TCP which is Transport Control Protocol. There is the possibility to increase the bytes for the Ethernet frame by using the Ethernet protocol. It can also increase the bytes of the whole package, resulting in an overhead per packet. Using them will provide enough bytes of data for typical haptic applications with respect to the available space per packet. Assuming a six-DOF kinematics with 16 bit (2 byte) resolution in their sensors and actuators, each packet has to carry only 12 bytes of data, with one packet for force-output and one for position-input. Even when considering that the data have to be extended with some additional overhead (address negotiations, status-information), this is still sufficient for many haptic applications. A disadvantage in using the Ethernet is given by the high efforts necessary for packet confection and protocol formulation, which would usually overload the computation power of standard microcontrollers. Additionally a high number of clients reduce the data rate within a network significantly. Using switches compensates this reduction to some extend. But the method of choice is usually given by an exclusive network for the haptic application.

Measurement Equipment and Multi-Functional Interface Cards

Measurement- and multi-function interface cards are a simple approach to interface to hardware designs. They are available for internal and external standard interfaces, such as PCMCIA, USB or even LAN. They are usually equipped with several standard sw-drivers optimized for their hardware capabilities. When considering a prototype design they should be considered in any case. Their biggest disadvantage is given by the data processing happening inside the hosting PC and within the restrictions of the operating system. Especially in combination with non-realtime operating systems like Windows the dynamics of controllers necessary for haptic applications may become not fast enough.

HIL-Systems

“Hardware In the Loop” (HIL)  systems were first used in control engineering and compensate the disadvantages from multifunctional cards for rapid prototyping and interfaces to haptic systems. HILs include a powerful controller with proprietary or open real-time operating system. The programs operated on these controllers have to be built on standard PCs and are transmitted as with any other microcontroller system too. Frequently the compilers allow programming with graphical programming language such as MatLab/Simlink or LabView. The processors of the HILs are connected via specialized bus-systems with variable peripheral components. Ranging from analogue and digital output over special bus- and actuator-interfaces a wide range of components is covered. HIL-systems are predestined for always the time-critical applications of haptics in design phase. But compared to other solutions they have a high price too.

5 Final Remarks About Interface Technology

The interface subordinates to the requirements of the system. Any realistic application and its required data rate can be covered with today’s standard components. This is a complete difference to the situation at the beginning of the 21st century. At this time highly specialized interfaces were designed for haptic devices, to cover the high requirements on data transmission rates. Accordingly even today commercial products with own ISA or PCI interface cards can be found on the market. Although the technical specifications are sufficient to fulfill the requirements, the first design and operation is far from being trivial.