Keywords

1 Introduction

For more than three decades [1] researchers around the world have supported the development of electronic skin (e-skin) technology. In robotics, the most important potential applications of e-skin are to improve the safety of human-machine collaboration and the agility of the robot [2]. E-skin devices currently being developed worldwide often allow the estimation of the value and location of tactile pressure [3]. There are also an increasing number of applications that allow the measurement of the proximity of human body parts to the e-skin surface [4]. Accurate testing and calibration is particularly important in light of the development in recent years of low-cost e-skin manufacturing techniques [5]. As the number of e-skin sensors increases, this problem becomes more relevant. As an example, previous study [6] performed an effective semi-automated calibration of e-skin, where the matrix studied had 16 rows and 18 columns, giving a total of 288 sensors. In order to calibrate and accurately test the e-skin, researchers typically construct a stand to apply a precise pressure to one selected e-skin sensor. The most naive approach is presented by [7], where reference touch pressure is implemented in the form of weights of known mass manually placed at selected locations on the surface of the e-skin. One can find more sophisticated approaches using XY-axis tables [8] or a Z-axis platform [9] with a load cell. In [10], an interesting test rig was described to implement e-skin stretching measurements using a motor and linear gearbox. In the case of the heterogeneous parameters of the e-skin sensors, the calibration procedures carried out on the presented benches would be labour-intensive and difficult to repeat in practice. In the literature reviewed, no publication was found presenting an e-skin test bench allowing universal and automated measurements. This manuscript describes a robotic workstation designed for the calibration and testing of e-skin fabricated as an array of multiple sensors with heterogeneous measurement parameters. The developed stand consists of an industrial standard robot, a reference force sensor and a customised tactile tool tip. Thanks to the use of an industrial robot, the repeatability of the experiments performed and the ease of automation of the measurement process are ensured. Experiments with e-skin performed at the stand may include, for example, testing several hundred regularly arranged e-skin sensors for their calibration, measuring the reaction to touch in terms of normal and lateral forces exerted on the e-skin, testing the estimation of the proximity of objects to e-skin skin, and many other scenarios possible with the use of an industrial robot. To carry out these tests, it is not necessary to change the elements of the robotic workstation significantly, but only to change the control programs. The paper describes the solutions used to integrate the robot control system, the reference sensor measurement system and the e-skin. Problems that may arise in this type of robotic application and the proposed way to solve them are discussed.

The paper is organised as follows. In Sect. 2, the developed robotic workstation and its software integration is presented. In Sect. 3, the example measurements results are given. Finally, Sect. 4 provides the summary and further investigation proposal.

2 Developed Robotic Workstation

The first part of this section describes the developed robotic workstation in terms of hardware. The use of an industrial robot as one of the components has a number of significant advantages. First and foremost, it allows universal use of the developed stand - it enables automation of the measurement acquisition process, ensures repeatability, precise setting of the pressure direction and, for example, testing the proximity of various types of objects to the e-skin due to the possibility of precise positioning of the robot. The next part of the section presents the workstation in terms of software integration. This task was the main challenge in developing the workstation. In order to ensure the required functionality of the developed robotic workstation and the possibility of using it, for example, for calibration, proximity testing, tactile touch testing, and other e-skin tests, it is necessary to ensure simultaneous robot movement and recording: tactile pressure measurements from the e-skin, measurements of the force exerted from reference device and TCP position measurements of the industrial robot. Measurements from different components should be synchronized in time. This chapter presents an exemplary e-skin test performed on the proposed workstation that meets all the above requirements.

2.1 Hardware Setup

The main components of the robotic workstation are a Fanuc LR Mate 200iC manipulator with a Mate R-30iA control cabinet (CC), an OnRobot Hex-e 6-axis reference device with a controller for measuring contact force, and the e-skin with a custom controller [3]. The central device integrating the measurements and communication with the devices is a general-purpose PC. Figure 1 shows pictures of the robotic workstation prepared for data acquisition.

Fig. 1.
figure 1

Illustrative representation of the developed robotic workstation [6]: 1. LRMate 200iC manipulator, 2. reference e-Hex sensor, 3. robot tool, 4. e-skin driver, 5. e-skin.

Full size image

As shown in Fig. 1, the Hex-e device (2) was attached to the end of the Fanuc robot kinematic chain (1). A custom tool tip with a compliant element (3) was then attached to the Hex-e. The use of the compliant element allows for a slower build-up of contact force with the robot’s standard positional control, and is an important part of the functionality of testing the tactile parameters of the e-skin. This is particularly important for highly sensitive tactile sensors (e.g. graphene-based). The e-skin (5) and its measurement controller (4) were placed within reach of the robot.

For control and measurement acquisition, all devices (1, 2 and 5) were connected to a PC using the appropriate drivers. A USB port was used for the e-skin, while the Fanuc robot and Hex-e device were connected via Ethernet to two separate network cards. The general connection diagram of the robotic workstation devices is shown in Fig. 2.

Fig. 2.
figure 2

Connection diagram of developed robotic workstation. [6].

Full size image

2.2 Software Integration

In order to meet the measurement objectives associated with the station, time-synchronised acquisition of measurements from the e-skin, the Hex-e device and the position recording of the Fanuc robot is necessary. A major problem in automating measurements on a robotic workstation was the software integration of communication with component devices. The following part of the manuscript shows how to calibrate the coordinate systems of an industrial robot and how to record the robot’s TCP position while integrated with the measurement systems of the e-skin and Hex-e reference device.

Robot Calibration. Before using the robotic workstation, it is necessary to define the tool and user frames for the manipulator. The tool coordinate frame is best defined so that the TCP (Tool Center Point) is unambiguous with the tactile tool tip being used. The user frame was then defined so that the origin of the coordinate frame was defined at one corner of the e-skin, while the direction of the X and Y axes are consistent with the columns and rows of the e-skin sensors. The tool and user coordinate frames were defined using three-point methods. With regard to the tool frame, it is important to easily physically identify the physical touch point of the tool tip. With regard to the user frame, it is necessary to identify physically with TCP the three points associated with the e-skin: the point defining the center of the corner sensor of the e-skin, the center of any sensor located in the x-axis direction, and any point located in the x-y plane of the e-skin. It is worth emphasizing that the TCP and the centers of the selected sensors should be easy to visually identify and available for physical touch.

TCP Position Acquisition. For the purpose of communicating with the Fanuc robot and recording its position, packages of the ROSFootnote 1 environment in the Indigo version running on the Linux distribution Kubuntu were used. For integration with the Fanuc robot, ROS-Industrial libraries were used. The process of running ROS for the Fanuc robot is complicated. In addition to the standard installation of ROS on a PC, it is necessary to install packages designed to run on the Fanuc CC. The installation process is based on compiling and loading the files into the CC with the Fanuc RoboGuide v7.70 (v7.70P/53 7DA7/53) software.

After network parameters configuration default ROS-Industrial package nodes allow to acquire only part of the status data from the Fanuc CC. This data includes information about joints angular positions. In order to capture the current position of the robot’s TCP, it is necessary to calculate it based on the kinematic parameters of the robot. In addition it is necessary to manually publish the custom tool frame tf transformation.

Hex-e and E-skin Acquisition. To implement measurements from the Hex-e device, a dedicated device controller was used along with a provided sample program in C++ implemented as a ROS node. In terms of measurements from the e-skin, the exact details of the measurement controller are described, for example, in [3]. From the programming perspective, multi-threaded integration of measurements with the ROS node responsible for measurements from Hex-e was implemented.

Example Application. The example developed application is designed to collect the data necessary for e-skin calibration. To do this, it is necessary to exert pressure on successive e-skin sensors and simultaneously record measurements from the e-skin and the reference Hex-e sensor. A recording of the robot’s position is not crucial for determining calibration, but it can be helpful in distinguishing between sensors, the loading and unloading phases of the exerted pressure, and thus in analyzing the hysteresis of e-skin sensors. The developed application generally consists of a PC-based ROS server program written in Python (1), a robot motion program (2) and ROS client for the robot both running on the CC (3), and a PC ROS node responsible for data collection (4). It is worth noting that there is a limitation on the number of simultaneously running programs in the robot CC. In order to simultaneously run the robot movement using (2) and the ROS client (3), it is necessary to set the “User Tasks” parameter in the CC configuration high enough. The robot’s motion program (2) is responsible for executing a repetitive pressure trajectory on successive e-skin sensors. A ROS client (3) is additionally run on the robot’s CC, responsible for sending messages about the robot’s current state to a server (1) running on a PC. The robot’s state, depending on the needs, can describe the robot’s position or, for example, operating states such as: downward motion, upward motion or travel between sensors. The ROS server on the PC (1) receives the robot status messages sent by the robot’s client program (3) and publishes a message on the corresponding ROS topic to record or not the measurement data. This is especially important to avoid large volumes of unnecessary measurement data being recorded. The node responsible for data recording (4) listens for messages on the ROS topic and starts recording measurement data of the e-skin, the Hex-e reference device and the robot’s position into files at appropriate times.

3 Example Results

Figure 3 shows selected measurement results from the operation of an example application on the developed robotic workstation. It is worth noting that, for example, data from the e-skin and the Hex-e device were acquired with different sampling frequencies: approximately 50 samples/s for the e-skin sensor and approximately 10 samples/s for the Hex-e sensor. Before using these measurements, it is necessary to solve the problem of different acquisition frequencies, e.g. by resampling. Figure 4 shows data from the selected sensor pressed during the test.

Fig. 3.
figure 3

Simultaneous acquisition of measurements from e-skin, Hex-e and Fanuc robot for five e-skin sensors.

Full size image
Fig. 4.
figure 4

Simultaneous acquisition of measurements from e-skin, Hex-e and Fanuc robot for one e-skin sensor.

Full size image

4 Summary

The manuscript describes the hardware and software framework for a e-skin versatile robotic workstation based on a standard Fanuc industrial robot. Using the example of the e-skin test described in Sect. 2.2, it was shown that the proposed workstation meets all the requirements to provide the required e-skin testing functionality in various scenarios. All the devices included in the station were integrated and work with the use of the PC and ROS programming system. Functional tests on the workstation showed that it could be used, for example, to calibrate the normal pressure of e-skin sensors characteristic. The application of the workstation, however, is much more versatile and can be used, for example, for testing sheer touch force e-skin reaction [3] or proximity estimation [4].

As outlined in the manuscript, the use of a standard industrial robot, e.g. Fanuc, is possible; however, its integration with the standard ROS programming system used increasingly in robotics development is a problem on its own. In the future, it is planned to replace Fanuc robot with the ES5 from EasyRobots companyFootnote 2, which has built-in integration with the ROS system.

The application of robotization in the field of e-skin testing leads to significant time savings (the acquisition of measurements from several hundred e-skin sensors can take several hours [6]) and improves the quality of measurements due to the repeatability achieved with an industrial robot. In order to further develop the workstation, it is worth solving the problem of manual calibration of the robot against the e-skin to further deepen the automation of the workstation and the associated benefits.