Haptic Teleoperation of UAV Equipped with Gamma-Ray Spectrometer for Detection and Identification of Radio-Active Materials in Industrial Plants
Large scale factories such as steel, wood, construction, recycling plants and landfills involve the procurement of raw material which may include radiating parts, that must be monitored, because potentially dangerous for workers. Manufacturing operations are carried out in unstructured environments, where fully autonomous unmanned aerial vehicle (UAV) inspection is hardly applicable. In this work we report on the development of a haptic teleoperated UAV for localization of radiation sources in industrial plants. Radiation sources can be localized and identified thanks to a novel CZT-based custom gamma-ray detector integrated on the UAV, providing light, compact, spectroscopic, and low power operation. UAV operation with a human in the loop allows an expert operator to focus on selected candidate areas, thereby optimizing short flight mission in face of the constrained acquisition times required by nuclear inspection. To cope with the reduced situational awareness of the remote operator, force feedback is exploited as an additional sensory channel. The developed prototype has been demonstrated both in relevant and operational environments.
9.1 Scientific and Industrial Motivations, Goals and Objectives
The presence of radioactive material is a real risk in different contexts such as steel, wood, construction, recycling industries, mines, and landfills. For example, not only secondary material of uranium mines results in a legacy radiological contamination , but also gold mining may produce the accumulation of radioactive materials in certain stages of the process . Also, radioactive contamination was found in wood and its products , in particular in wood pellet . Another source of contamination comes from the illicit dropout of nuclear sources that are molten in scrap metal . Indeed, not in all large-scale factories or landscapes the control of the materials at the entrance is compulsory, and, even when such control nominally exists, the monitoring procedures have become only recently operative.
At present, if a large industrial area has to be monitored for the presence of radioactive material, then the search is mainly led by a human operator carrying proper instrumentation, with the unavoidable exposure of the operator to the radiation. Moreover, stacks of materials to be monitored in most cases are too large or too high for a direct monitoring by a human operator so that the human operator is inhibited to map all the zones of the stack.
In this context, the exploitation of an unmanned aerial vehicle (UAV) equipped with a radiation detector permits, at the same time, to guarantee the safety of the operator that can guide the UAV from a secure zone, and the monitoring of an area that can be reached only by an UAV.
The research field on radiation detection using robotic systems comprises two main topics, i.e. localization and mapping. In particular, localization addresses the problem of finding a point source of radiation using the perceived radiation distribution as a guide in the search behaviour of the robot. Radiation mapping is focused on the problem of generating a complete map of an area. Dealing with illicit or unforeseen radioactive materials in unknown locations involves major safety hazards, hence procedures and techniques for reducing human exposure time and radiation risks can provide benefits of utmost importance, while also serving as a deterrent measure.
Teleoperation of aerial vehicles is complex due to the lack of situation awareness of the operator. Indeed, when only visual feedback is available to the operator, such limited feedback is often inadequate to support critical remote operations. Indeed, the information provided by direct sight or through an on-board camera may not be sufficient due to the limited field of view. Moreover, time delays in the aerial environment can make teleoperation a difficult task. For example, an obstacle outside the field of view cannot be detected by the operator, which may lead to a collision.
Hence, it is crucial to investigate alternative strategies for aerial teleoperation, which provide additional information sources to the operator by exploiting signals from sensors. Such information is helpful to increase the perception of the environment constraints, which can increase the situation awareness of the operator and improve efficiency and task performance.
Using haptic teleoperation with force feedback is a promising solution for aerial teleoperation. In haptic teleoperation the control system feeds external forces to a haptic device.
In this paper a unique platform is presented for studying the potential of non-conventional human-robot interaction techniques in the context of aerial radiation localization. The study was concerned with the general goal of developing technologies for improving the interaction between operator and machines in challenging industrial contexts by the exploitation of a haptic user interface for remote control of an Unmanned Aerial vehicle (UAV). By using a single bilateral, haptic interface the user is able to provide 3D motion commands to the UAV and receive at the same time an informative force feedback via the interface. The advantages obtained with this approach in the context of radiation detection will be discussed in the following of this paper. Moreover, a novel CdZnTe-based custom radiation detector is integrated on board of the UAV, so that radiation sources can be localized and identified without a direct exposure of the human operator. By means of CdZnTe crystals, solid-state radiation detectors can be realized such that their limited weight and power consumption enable sensor integration even in UAVs offering a small payload. Moreover, spectroscopic detectors could be realized in a similar technology, with the advantage of enabling localization and identification of the radiation source at once. By combining haptic guidance and CdZnTe-based custom detector, novel teleoperation technologies are developed to reduce the risks connected to dangerous processes in large-scale industrial environments.
Since the final goal was to make the prototype available for future exploitation by third parties (public agencies for environment monitoring, other territorial agencies, private companies), the UAV equipped with the radiation sensor and the haptic guidance system has undergone the protocol required to achieve the Italian certification for standard operations. Also, operators have been required to obtain the licence for piloting the UAV. Achieving the national certification for the UAV, as well as proper driving licences, has been an important milestone towards potential market applications of the system.
The prototype has been demonstrated in relevant and operational environments. These experiments have been carried out with the cooperation of the local agency for environmental control, ARPAE Emilia Romagna, and Emiltest s.r.l. in Gossolengo, Piacenza, Italy. The aim of these experiments was to demonstrate the effective sensitivity of the system to weak nuclear sources and the localization capability of the prototype. A second set of experiments has been carried out in operational environments. Flights were performed in a waste disposal landfill in Novellara, Reggio Emilia, Italy, monitoring the specific radiation levels and checking for any hidden radiating source.
9.2 State of the Art
The use of UAVs for environmental nuclear radiation monitoring was investigated in several works [1, 6, 7, 8, 9, 10, 11, 12, 13] either by exploiting standard remote controllers or by specifying pre-programmed flight missions. None of previous works investigated haptic teleoperation for nuclear radiation detection. Indeed, so far haptic teleoperation of UAVs has been investigated exclusively to assist the operator for collision avoidance as reported in .
A UAV system was proposed in  for air quality control, water pollution monitoring and radiation leakage detection using a cell biosensor system. Okuyama et al.  reported the experimentation with an autonomous helicopter to measure radiation data and provide real-time data transmission to a ground station, including images. Boudergui et al.  carried out a preliminary evaluation of a teleoperated UAV equipped with a CdZnTe-based sensor and a gamma camera for nuclear and radiological risk characterization in indoor environments. Operating an UAV in outdoor environment poses significantly different challenges in terms of situational awareness and flight mission optimization. In  and  fixed wing UAVs capable of flying at high altitude and high speed were developed for radiation detection. Experiments were performed around the Fukushima Daiichi nuclear power plant in [11, 12, 13] where the radioactive caesium deposition on the ground was successfully measured. In other works [15, 16, 17] experiments in simulated environments were reported. In  a simulation of an autonomous helicopter was developed for radiation detection and imaging. In  a multiple UAVs radiation contour mapping task was simulated with formation flight control. In  a method for multiple source localization and radiation contour mapping using a UAV was proposed and evaluated with numerical simulations.
Several gamma ray detectors are available on the market . The most exploited ones are Geiger-Müller detectors. However, these gas detectors show little sensitivity, are fragile (they are basically gas-filled glass tubes), and do not provide spectroscopic performances. Scintillators are also quite diffused. The light produced by the scintillating material can be picked up by photodiodes instead of fragile phototubes. However, the energy resolution of scintillators is not satisfactory, especially if low energy gamma rays are detected. On the market, several semiconductor-based detectors are available. Silicon detectors show good spectroscopic performances, however they provide limited stopping power for energy larger than few tenths of KeV. Germanium detectors show excellent spectroscopic performance, however they require cryogenic cooling, and thus they are not easily integrated on a UAV.
Due to the fact that the detector must work at room temperature with excellent energy resolution in a wide energy range, the best candidate material is CdZnTe. Also, the high electrical resistivity of CdZnTe crystals ensures low power consumption.
For these reasons, CZT detectors can easily integrated on a UAV. In  a teleoperated quad-rotor had previously proposed that performs nuclear and radiological risk characterization using a CdZnTe sensor and a gamma camera for security purposes. The system was proposed mainly for security applications, thus in small indoor contexts. Moreover, the system was only proposed, and its actual performance was not shown.
9.3 Problem Statement and Proposed Approach
Another important aspect is the use of a CdZnTe-based gamma ray spectrometer. Since the detector must localize and identify nuclear sources, and must be mounted and carried by the UAV, the detector characteristics must fulfil the following requirements: (i) overall weight (detector, read out electronics, and batteries) lower than 0.5 kg, (ii) operate at room temperature, (iii) low power consumption (battery operated for 1 h), (iv) large energy range: 10 keV–1.5 MeV, (v) good energy resolution (<3% @662 keV), (vi) high sensitivity (>1000 cps for a 57Co source of 1 mSv/year at 2 m from the detector), (vii) large field of view. The extended autonomy of the gamma-ray sensor (at least 1 h) will enable multiple UAV missions by simply swapping the battery of the UAV, without requiring unmounting and reconfiguration of the sensor unit.
The detector must work in a wide energy range (10 keV–1.5 MeV) to reveal the main potential nuclear materials. High energy gamma can be efficiently stopped only by thick detectors. This means that in any case, a several millimetre thick CdZnTe detector must be employed. Nowadays, on the market, CdZnTe detectors as large as 20 × 20 × 6 mm3 are available, thus it makes sense to evaluate whether the volume of such detector is large enough for the scope of the proposed application.
In order to evaluate the required sensitivity it should be recalled that the aim is to detect nuclear sources in industrial plants or material stocking plants whose intensity can be dangerous for the health of operators or workers. Italian regulations , in accordance with the recommendations of the International Commission on Radiological Protection , set the limit of exposition doses for workers to 1 mSv/year. It is therefore reasonable for the UAV to look for nuclear sources whose activity is enough to expose a worker standing at one meter distance to such a dose. In order to preserve the integrity of the UAV, it is better to keep the UAV at a height of about one meter from the ground, even though in most situations the UAV can fly at lower height.
Counts per second measured by a 20 × 20 × 6 mm3 CdZnTe detector at 1 m from nuclear sources potentially dangerous for a worker
Source activity (MBq)
Counts per second
9.4 Developed Technologies, Methodologies and Tools
The UAV was also equipped with a remotely Flight Terminator and with a parachute. Finally, the UAV was equipped with a video camera pointing to the ground to evaluate offline the error between the actual location of the radiation substance and the location estimated by the operator through the haptic feedback.
The haptic device is a 3 degrees of freedom Novint Falcon (top left inset in Fig. 9.1), which features a position resolution of 0.0635 mm, a range of motion of about 10 cm3 and a maximum force feedback of 10 N.
The teleoperation approach is based on an impedance control mode where the user specifies the horizontal heading direction of the UAV by moving the tool point of the haptic device. In particular, a movement of the haptic device causes the change of the UAV waypoint , and so the direction of the UAV movement.
If the tool point of the haptic device is within a fixed small range from the centre of the haptic reference frame the waypoint is not updated and the UAV hovers. The UAV altitude set point is provided by the standard controller and can be changed at any time.
A force feedback is provided to the operator as a 2D vector on the horizontal plane of the haptic workspace, acting as a basin of attraction with quadratic profile, causing the UAV to be attracted by the radiation source. Indeed, as the pilot moves away from radiation source, a force is generated towards the maximum of the detected radiation whose position is updated over time.
The number of counts is integrated over a time frame of 2 s. Outside the zone of attraction, no force feedback is calculated. Force feedback can be reset by simply acting on the haptic device, so that multiple concentrated radiation sources can be localized one by one.
Transmission of measured radioactivity data from the UAV to the ground station is performed through a dedicated long-range 5 GHz WiFi connection.
For sensor manufacturing, standard pixel detectors from REDLEN were acquired, characterized with certified spectroscopic properties. Four 5 × 6 × 20 mm3 samples were obtained by carefully sawing. All the six surfaces of each detector were polished. In fact, it is known that also the preparation of lateral surfaces is important in order to reduce leakage currents. Polishing consists of two main steps, the first one with abrasive paper and the second one involving alumina powder suspension. The strip grid configuration shown in Fig. 9.4 was obtained using a photolithographic process and depositing gold contacts by the electroless technique with alcoholic solutions . This type of contacts ensures good mechanical stability and blocking current-voltage characteristics, thus limiting dark current. This is very important to get good spectroscopy detectors. The detectors were bonded on Teflon-based substrates in order to ensure optimal electrical insulation using silver epoxy and gold wirings .
A battery provides the required power for the detector and the read out electronics. The system consumption is relatively low, so that a small battery is enough to power the detectors for 8 h operations. A low noise DC-DC converter was used to bias the detectors with the required 600 V. The read out electronic chain was constituted for each of the four detectors by a charge sensitive preamplifier, a shaping amplifier, and a peak detector. Signals from the four detectors were collected by a microcontroller converting the analogue signal into a digital signal and operating as a multichannel analyser to generate the energy spectra. These data are then sent to the ground station by the Intel-Galileo system described before.
The detectors and the electronics were housed in a 3D printed plastic box (see Fig. 9.2). The plastic box was covered with aluminium tape to create a noise shielding. The total weight of the box, with the detector and the electronics was about 300 g, much lower than the UAV maximum payload.
9.5 Testing and Validation of Results
The first test of the prototype has been carried out in laboratory and concerned the capability of the detector integrated on the UAV to collect spectra of a number of nuclear sources (Sect. 9.5.1). Then, the prototype has been tested in a relevant environment (Sect. 9.5.2). The aim of these experiments was to demonstrate the effective sensitivity of the system to weak nuclear sources and the localization capability of the prototype. Finally, a set of experiments has been carried out in operational environments (Sect. 9.5.3). Flights were performed in a waste disposal landfill checking for any hidden radiating source.
9.5.1 Laboratory Test
It is worth mentioning that the drift strip detector also showed a two times larger sensitivity with respect to the other.
Preliminary experiments for validation of the haptic interface have been performed using a small, easily manoeuvrable UAV not equipped with the actual gamma detector. The presence of a nuclear radiating source on the ground was simulated. The operator, indeed, was unaware of the actual location of the simulated sources. These preliminary experiments allowed to establish the effectiveness of the approach as well as to refine the haptic interaction modality  and verify the stability of UAV teleoperation at the operating speed required for radiation monitoring purposes.
Experiments showed that the haptic-based teleoperation system helps the operator in focusing the search in the areas close to the radiating sources, thanks to the better support received through the haptic feedback channel.
9.5.2 Test of the Prototype in a Relevant Environment
The UAV prototype was validated in field experiments in two outdoor environments under the supervision of the regional environmental protection agency. The first location was an industrial plant in Gossolengo, Italy. A nuclear source (192Ir) was located in the service area. This source is routinely used for non-destructive testing of large metallic elements soldering. Because of the large intensity of the nuclear source, the UAV can detect the gamma radiation in a large area. The nuclear source was inserted into a vertical lead container (to prevent lateral radiation emission) placed on the ground at the centre of a target with concentric circles (0.5 m radius step size). Preliminary experiments have been performed with prior knowledge of the position of the nuclear source that was placed in direct sight of the operator. These experiments were carried out to actually test the functionality of the whole prototype and of the haptic interface in particular. As the UAV entered inside the radiation basin, the detector started to reveal gamma events and the pilot could feel a force on the haptic interface. Thus, the pilot flew the UAV back and forth several times in order to collect data on the whole system response.
9.5.3 Test of the Prototype in Operational Environment
Further experiments were carried at a location that was chosen to represent a possible operational environment for the exploitation of the prototype. With this in mind, we performed flight experiments in an inter-municipal landfill in Novellara (RE), Italy, used for the disposal of waste materials. The facility is classified as a type B landfill which in principle excludes the presence hazardous waste. However, a recurrent environmental issue is whether radioactive material has been illicitly hidden in landfills, and hence landfill monitoring is indeed a potential operational environment for the drone. Clearly, in these environments the challenge in searching for nuclear sources it is not only given by the threat of the exposure to gamma radiation, but also by the harsh environment itself.
The environment of the landfill in Novellara was not flat, hence multiple exploration trials have been performed at different heights by changing the UAV altitude set-point.
The haptic user interface was also evaluated in field experiments in comparison to a standard mode of operation where the operator could only see sensor feedback from the detector by looking at a monitor screen. Results, reported in , indicated that without force feedback the operator had to spend about 23% of the time looking at the screen. On the other hand, the operator remained steadily focused on the flying UAV when the haptic feedback was turned on. This demonstrated that the use of force feedback can help to reduce the operator workload and minimize the risk of dangerous events like collisions and crashes.
9.6 Conclusions and Future Research
The developed prototype proved to be an effective tool for localization and identification of nuclear material dispersed in the environment or illicitly stocked in industrial plants. The effectiveness of the approach was demonstrated in the lab, in a relevant environment in the presence of an intense 192Ir nuclear source located in a service area, and in an operational environment consisting of an inter-municipal waste disposal landfill.
The prototype integrates a UAV with an airborne innovative CdZnTe-based gamma ray detector, able to detect with high sensitivity all the main nuclear contaminants. Thanks to the spectroscopic grade of the detector, identification of the localized nuclear source was also demonstrated in the project.
A haptic interface was adopted to drive the UAV, exploiting the real time counting rate experienced by the detector and an innovative software tool, fully compatible with the standard piloting software of the UAV. The pilot experienced on the haptic interface an attractive force that drove him towards the nuclear source. This teleoperation solution not only reduces the localization time, but also enables the operator to drive the UAV without missing the continuous view of the vehicle.
The results of this research have been presented at several conferences and the prototype has been shown at several national and international events An International patent has been filed  disclosing the main achievements of this research. Moreover, the realized prototype is now ready for exploitation by third parties (e.g. ARPA Emilia-Romagna, other territorial agencies, private companies), under the responsibility and control of IMEM-CNR and University of Parma. A web site was created2 illustrating the main features of the prototype, and the experiments carried out. Through the website it is also possible to access the procedure to be followed by third parties and stakeholders in order to take advantage of this new prototype.
This work has been funded by the Italian Ministry of Education, Universities and Research (MIUR) under the Flagship Project “Factories of the Future—Italy” (Progetto Bandiera “La Fabbrica del Futuro”) , Sottoprogetto 2, research projects “XDrone” and “X-Drone2”.
We thank ARPA Emilia Romagna and in particular Roberto Sogni for providing supervision and support for the experimental evaluation. We also thank “Emiltest srl Controlli Non Distruttivi” in Gossolengo, Piacenza, Italy and in particular Filippo Panciroli for granting access to their facility. We also thank Giancarla Rossetti for helping us with the authorization procedures for moving the nuclear source. Finally, we want to thank S.a.ba.r. S.p.A. for hosting our experiments on the landfill.
- 2.Wendel G (1998) Radioactivity in mines and mine water-sources and mechanisms. J South Afr Inst Min Metall 98(2):87–92Google Scholar
- 5.Ortiz P, Friedrich V, Wheatley J et al (1999) Lost and found danger—orphan radiation sources raise global concern. IAES Bull 41:18–25Google Scholar
- 8.Boudergui K et al (2011) Development of a drone equipped with optimized sensors for nuclear and radiological risk characterization. In: 2nd international conference on advancements in nuclear instrumentation measurement methods and their applications (ANIMMA). IEEE, pp 1–9Google Scholar
- 9.Kurvinen K, Smolander P, Pöllänen R et al (2005) Design of a radiation surveillance unit for an unmanned aerial vehicle. J Environ Radioact 67(2):340–344Google Scholar
- 18.Knoll Glenn F (2000) Radiation detection and measurement, 3rd edn. Wiley, USAGoogle Scholar
- 20.Camarda GS et al (2007) CdZnTe room-temperature semiconductor gamma-ray detector for national-security applications. In: Systems, applications and technology conference, LISAT 2007. IEEE Long Island, INSPEC, Accession Number: 9830508Google Scholar
- 22.D.Lgs. 230/95 e s.m.i. http://www.repertoriosalute.it/wp-content/uploads/2016/11/dlgs230-95.pdf
- 23.International Commission on Radiological Protection. http://www.icrp.org
- 24.Micconi G et al (2015) Haptic guided UAV for detection of radiation sources in outdoor environments. In: Proceedings of the 3rd RED-UAS 2015 workshop on research, education and development of unmanned aerial systems, Cancun, Mexico, 23–25 Nov 2015Google Scholar
- 25.Kuvvetli I et al (2014) A 3D CZT high resolution detector for x- and gamma-ray astronomy. In: SPIE—the international society for optical engineering. 2014. 9154, art. n.:91540XGoogle Scholar
- 26.Bettelli M et al (2016) CdZnTe detector prototype for boron imaging by SPECT during BNCT treatment: simulations and measurements in a neutron field. In: 2016 IEEE NSS/MIC/RTSD, Strasbourg, Nov 2017, R01-7Google Scholar
- 29.Micconi G et al (2016) Evaluation of a haptic interface for UAV teleoperation in detection of radiation sources. In: 18th Mediterranean electrotechnical conference: intelligent and efficient technologies and services for the citizen, MELECON 2016Google Scholar
- 30.Aleotti J, Micconi G, Caselli S, Benassi G, Zambelli N, Calestani D, Zanichelli M, Bettelli M, Zappettini A (2015) Unmanned aerial vehicle equipped with spectroscopic CdZnTe detector for detection and identification of radiological and nuclear material. In: 2015 IEEE nuclear science symposium and medical imaging conference (NSS/MIC), pp 1–5Google Scholar
- 31.Aleotti J et al (2017) A system and a relative method for detecting polluting substances using a remotely piloted vehicle from a haptic command device. Patent WO 2017055962 A1Google Scholar
- 32.Terkaj W, Tolio T (2019) The Italian flagship project: factories of the future. In: Tolio T, Copani G, Terkaj W (eds) Factories of the future. SpringerGoogle Scholar
Open Access This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.
The images or other third party material in this chapter are included in the chapter's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.