Introduction

In the last decade, there has been a significant interest in the security of ports and marine infrastructures due to the growing threats against civilian targets. The vast scale of the European maritime region and the complexity of the challenges associated with ensuring successful maritime surveillance require a global approach to maritime security. As a result, projects involving different countries at European and NATO levels have been set up to fill the technological gap identified in this branch and to provide new methodologies. Underwater surveillance has traditionally been carried out by means of manned solutions. New maritime capabilities which include the way to conduct surveillance and protection of ports and coastal waters are treated within the Harbour & Maritime Surveillance and Protection (HARMPRO) project [1], implemented within the Permanent Structured Cooperation (PESCO) framework. In the same project framework, the Maritime Unmanned Anti-Submarine System (MUSAS) project will aim to develop an innovative system based on artificial intelligence for anti-submarine warfare that will strengthen the security of underwater infrastructures, providing a rapid reaction of sufficient measures of force to intruders. The Programme of Research for Defense Against Terrorism (DAT) provides measures to reinforce the organisation and coordination of actions to deal with terrorist incidents [2]. This initiative includes programmes aimed at reducing the vulnerability of staff and facilities in all circumstances. Harbour operations are vital to the world economy and need to be as secure and safe as possible. For this reason, global institutions have been exploring various technologies to improve maritime security.

In this framework, a key role is played by robotics, and more specifically by unmanned vehicles (UxV), that certainly are a game changing technology in this field. The involvement of UxV within a surveillance system helps in replacing human low level activities, giving to the human operator a high value information for higher decision level. Jointly working with other observation and manned systems, UxV are evolving into reliable, endurance, persistent and cost-effective surveillance solutions. Much has been debated for the use of these technologies, both at the scientific and operational level, highlighting the current potential possibilities and mapping a realistic outlook for their future. Such analyses are continually changing as, on the one hand, the technological development strengthens the roles already achieved in nowadays systems, making it possible to hypothesise more complex activities; on the other hand, scientific advancements provide creative potentials opening the way to new theoretical scenarios. The involvement of robotic systems in operating environments, whether military or civilian, has already begun to shift the methods that have historically been adopted in carrying out missions, such as detection, identification and surveillance. The Navy Unmanned Undersea Vehicle Master Plan [3] is a valuable document that helps to map the employment scenario of the unmanned underwater vehicles (UUV), evaluating their capabilities and analysing their strengths, allowing for an accurate planning of acquisitions and technological research for many purposes. The Master Plan recognises the UUV component as a potential power multiplier in the operational scenario and a risk reduction tool for human operators. UUV capabilities allow end users to approach different task areas depending on the specific applications. The intelligence, surveillance and reconnaissance (ISR) abilities of UUVs can increase the capabilities currently running on systems historically operated by humans, expanding power and identification in complex environments, in high-risk areas for humans or even in shallow water where conventional systems are unable to operate. The relevance of the robotics impact on the field of maritime surveillance is also due to the capability of the UxV of hosting onboard, as payload, a variety of sensors as, e.g. sonars, hydrophones, magnetometers and optical or infrared cameras. Through these devices, the vehicles are able to conduct a rapid search task aimed at the identification and localisation of underwater targets and objects in confined areas, such as piers, within harbour areas or specific regions of interest.

This paper offers a review of recent advancements in the role of robotics in underwater surveillance, i.e. the control and monitoring of an area of competence aimed at identifying potential threats in support of homeland defence, antiterrorism, force protection and Explosive Ordnance Disposal (EOD). The threats are asymmetric, and they can involve the use of surprises in both their tactical and strategic aspects, unpredictable behaviours and the use of no-conventional weapons in no-conventional ways.

A revision of the recent literature on this topic is here proposed developing the discussion of previous related analyses [4••]. This work is to be intended as a critical guide to the recent prolific bibliography on the topic, giving also a general context in terms of mission requirements.

The paper is organised in the following into two sections that address the involvement of robots in underwater surveillance under two complementary points of view: “Robots for Seabed Exploration” is related to use of robots for seabed surveillance, as typical in Mine Counter Measure (MCM) operations and analogous tasks; “Intrusion Detection Robots” is focused on the role of robots in systematic patrolling for intruder detection. “Main Research Frontiers in Underwater Surveillance,” on the basis of the critical review of literature of the previous sections, proposes a discussion of the main research frontiers in underwater surveillance. “Conclusions” offers some general conclusions.

Robots for Seabed Exploration

One core area of underwater surveillance includes the capability of identifying and localising explosive and hazard devices. This family includes improvised explosive devices (IEDs), naval mines and unexploded ordnance (UXO). The underwater IED [5] represents a growing terrorist threat and is defined as an explosive device that is placed or fabricated in an improvised manner, by an irregular force created to hit a strategic or tactical goal. The IED risk is particularly high in Confined and Shallow Waters (CSW), especially in harbours, anchorages and routes to approach harbours. Underwater mines are defined as “explosive devices laid in the water by regular forces, with the intention of damaging or sinking ships or deterring shipping from entering an area” [6]. They can be employed in a minefield or individually moored, floating, ground or buried.

UXOs are old unexploded artefacts resting over or in the seafloor. They do not represent a real problem for already existing structures, but they must be taken into account before building a new infrastructure or in operation of clearing a new area. Clearing sea from UXO is a new challenging task for governments. A rich literature in UXO detection explains the multiple advantages of using magnetic sensors [7]; recently, considering the heterogeneous nature of UXO, there are relevant studies that consider also acoustic sensors [8] and the reusage of already existing systems and algorithms for MCM purposes in order to perform UXO detection [9].

Multiple and combined approaches can be used in order to contrast the mentioned threats, which can range from intelligence to simply selecting different routes during navigation. Surely, the inspection and the surveillance of the sea bottom are a fundamental step for an area to be considered safe. Traditionally, this has been done by using mine hunter/minesweeper vessels or deploying divers. Thus, man and expensive specific assets were used inside a potentially dangerous area. The usage of robots as UUVs has its main goal in reducing the risk for human operators and keeping them out of the minefield: no one knows how a mine is set and triggered; the only way to be safe is to stay far away from the explosive.

Recently, the growing attention in artificial intelligence (AI) opened new scenarios in the usage of autonomous underwater vehicles (AUVs) [10]. They can accomplish higher level tasks and take some decisions without the supervision of a human operator. This has a double meaning: on the one hand, it allows to cover the communication gap [11•] that affects the underwater domain by limiting the amount of data that the vehicle requires to exchange; on the other hand, it allows to have a persistent control with the same level of attention [12, 13] without suffering from typical human stress conditions.

UUVs can carry a plenty variety of sensors [14]. These can range from acoustic ones, such as side scan sonar (SSS) and synthetic aperture sonar (SAS), which create a side-looking image of the sea bottom, to an echo sounder (Single and MultiBeam) that creates a 3D map.

Sub Bottom Profiler (SBP) can also be used for the inspection of the sea bottom sedimentation. Furthermore, forward looking sonar (FLS) and acoustic cameras represent an alternative solution depending on the specific task. Magnetic sensors such as magnetometers and optical cameras and laser scanner may be used in very specific applications.

Imaging sonar (SAS, SSS and FLS) operations using UUVs can employ image processing techniques to quickly and accurately find targets. Convolutional neural network (CNNs) can be considered the state-of-the-art performance of image classification tasks also in the maritime domain. Many authors propose their usage for both side-looking sonar [15, 16] and FLS [17] achieving good results with well-trained networks.A considerable problem of the underwater domain is the difficulty of obtaining precisely geo-referenced images collected by a sensor and, consequently, the difficulty of subsequently locating found objects. The lack of GPS into the water makes necessary the use of solutions [18, 19] based on one or more surface gateways and systems of underwater positioning and communication such as ultra-short baseline (USBL), short baseline (SBL) or long baseline (LBL) techniques. While underwater localisation is offering promising results, a useful technique is the change detection (CD) method, based on the process of identifying objects or other phenomena of interest as temporal differences by observing a scene at different times [20,21,22]. In mine hunting, CD is an important application. Reference sonar imagery of strategic ports, inlets or sea lines of communications are recorded during route surveys, when the seafloor is assumed to be free of mines. After a new survey, mines are recognised as objects that are only present in the current imagery reducing the possibility of false detection and the time to analyse data.

Intrusion Detection Robots

Surveillance is one of the ways in which military and civilian implementations are very much intertwined and in several cases overlapping. These activities concern, for instance, coastal infrastructures, both military and civilian, including shipyards and offshore structures. Intruder threats can be divers or midget submarines. In addition, UxV should also be considered as a potential threat. Indeed, audio or image data collected by unmanned vehicles would be useful in strategic warfare against waterside facilities. Furthermore, they can potentially carry explosive payloads and may be used to target piers, merchant and military boats and docks [23], offering a significant advantage in asymmetric threat. Furthermore, UUVs could be potential intruders due to the smaller signature and target strength than the ones of a diver, and they are able to move faster, remaining below the detection threshold of active sonars at longer ranges [24].

There are currently different kinds of surveillance systems and strategies under investigation. Since at least the early 1980s, various underwater security programmes have been implemented to counter the risks in ports [25]. They mostly rely on the use of high-frequency active sonar, with detection ranges of less than 1 km. Traditionally, underwater surveillance has been conducted by manned platforms equipped with advanced and expensive sensors, which can be costly and with high workload [26]. The UUVs can play a leading role in this context, cooperating with surface and underwater assets. The deployment of a team of unmanned surface vehicles (USV) to perform patrolling surveys may be very effective to improve the performance of a harbour security system [27]. A solution to the issue of intercepting threats has been proposed by Simetti et al. [28] based on the optimisation of interception time and the position by using a team of USVs conducting surveillance operations. Nardi et al. [29] modelled as a potential game the issue of controlling a team of UUVs used as a defence system against asymmetrical threats, proposing a tool for designing the size of the team based on intruders’ characteristics. In the H2020 WiMUST project framework [30], the design and experimental demonstration of a distributed reconfigurable autonomous underwater array has been addressed [31••]. The system consists of a team of cooperative robots equipped with hydrophone streamers for collecting seabed data, but it would support a significant range of applications, including acoustic surveillance. The use of cooperative robotics networks for underwater surveillance applications has been pursued by the NATO Centre for Maritime Research and Experimentation (CMRE). In the experiment presented by Ferri et al. [32•], two cooperative UUVs equipped with acoustic sensors produced tracks onboard that were classified in real time and shared between the two UUVs and a ship acting as a high value asset (HVA), totally integrated into the network, in order to create a tactical picture. Since electromagnetic waves are strongly attenuated into the water, underwater surveillance based on acoustic waves becomes the most commonly used way to monitor the underwater environment. An example of underwater surveillance scenario is illustrated in Fig. 1.

Fig. 1
figure 1

A conceptual scenario of underwater surveillance based on unmanned systems. A network of UxV, each one carrying one or more specific payloads and fulfilling a specific mission, communicates, collects data and performs real-time monitoring. The communication infrastructure between the underwater and air environment is guaranteed by USVs acting as gateway buoys, i.e. equipped with acoustic and radio modems. The support ship is represented by NRV Alliance, a NATO research vessel used to conduct a wide range of oceanographic activities and that can act as C2S. In the sketch, UUVs are equipped with vector sensors and/or acoustic arrays for intruders’ detection. Instead, some of them are equipped with sidescan sonar payloads for seabed exploration

Full size image

Some solutions for localisation of underwater sources in surveillance applications employ acoustic vector sensors, which are rapidly growing in popularity and attention, since they can enhance the monitoring capabilities compared with the traditional hydrophones, thanks to their ability to provide directional information of noisy targets [33]. Furthermore, they are small, lightweight, low power consumption and suitable to be used on UUVs: potentially, they can provide the same information of a towed hydrophone streamer through a compact sensor easy to install and deploy from a marine robot. The current drawback of vector sensors is the great sensitivity to robot motion, which has to be compensated in the raw data. The development of a dual accelerometer vector sensor mounted on the MEDUSA class AUV [34] is described by Santos et al. [35]. Although designed for geophysical surveys, the concept can be exploited for acoustic surveillance applications. The CMRE integrated a commercial vector sensor [36] into the Slocum buoyancy glider [37], with the aim to demonstrate that UUVs can produce accurate target bearings. The obtained results potentially represent a starting point for the use of multiple gliders equipped with advanced sensors integrated into robotic surveillance networks. Experimental results on bearing estimation using a vector sensor from DIFAR sonobuoy [38] mounted on the eFolaga hybrid AUV [39] are presented by Terracciano et al. [40]. The developed system shows very interesting performance in particular in terms of bearing estimation of multiple targets, providing an additional new tool for robotics underwater surveillance. Further approaches introduced include the use of volumetric arrays, as in the case of the CMRE which developed one for looking in all directions. It has been mounted on the nose of the Slocum glider to address the issue of marine traffic monitoring in a given maritime area [41].

Main Research Frontiers in Underwater Surveillance

Considering the problems posed by underwater communications and networking, depicted in the previous sections, the latest research direction is to leave static and monolithic inbuilt modems and connectivity stacks for software-defined and cognitive, adaptable solutions. The use of Software-Defined Open-Architecture Modem (SDOAM) brings the great advantage of being able to calibrate all communication parameters (e.g. waveforms, frequencies, encodings) according to the conditions of the acoustic channel, which is strongly non-linear and variable [42]. Recent works have shown how it is possible to adapt the operation of such modems according to network requirements and vehicle mission profiles, taking into account specific channel metrics [43], even within a very short time of reaction [44]. In order to use software-defined modems effectively, a cognitive and adaptable approach is also necessary in the upper layers of the communication protocol stack. In the recent past, several architectural solutions for networks have been proposed and developed that focus on fine synchronisation between the tasks performed by the various layers [45]. These solutions have also been successfully implemented on AUVs and tested during sea trials [46, 47]. An example of a research centre that dedicated effort to the study, development and dissemination of this technology since 2011 is the CMRE. Recently, CMRE published an article outlining the main development and implementation activities of SDOAM and future directions [48]. From this solid starting point, the CMRE communication stack is evolving into a fully cognitive communication architecture (CCA) to establish an intelligent, adaptable and safe underwater network [49].

In addition, the scientific community, military end users and industry are increasingly paying attention to aspects relating to multi-vehicle operations, manned-unmanned interoperability and (cyber-)security [11•]. It is necessary to underline the fact that interoperability [50] depends on many interconnected topics, such as the abovementioned Adaptive Autonomous Communications Networks, the Command and Control Systems (C2S) of robotics system-of-systems and the continuous standardisation of the interoperability. From the point of view of interoperability standardisation, there are two very significant NATO STANAGs which deserve to be listed in this paper: 4586 and 4748. First, STANAG 4586 [51] is the current standard for unmanned aerial vehicles (UAVs) [52], which is the starting point for a new study aimed at establishing the “joint” multi-domain control station (MDCS) standard, i.e. unmanned air, land and maritime systems. In the future, the MDCS Working Group—a combination of industry and government representatives—will formalise a revised version of STANAG 4586. Second, CMRE, together with academia and industry, has developed a standard underwater communication protocol known as JANUS [53], recently ratified by NATO as STANAG 4748 [54]. This is the first internationally recognised and open-access underwater digital communication standard [55], already widely validated at sea for automatic identification system (AIS) data exchange and for the transfer of meteorological and oceanographic data to submarines [56,57,58]. Thanks to the use of a standardised approach, it will be possible to increase the functional level of Maritime Situational Awareness (MSA) for robotic surveillance by using heterogeneous and hybrid systems from different manufacturers, as long as they comply with the JANUS standard.

The fact that an underwater acoustic channel is inherently broadcast poses significant cyber-security risks in sensitive tasks, such as autonomous robotic surveillance, because an attacker is easily able to disrupt or intercept communications within such a network. The use of spread spectrum (SS) signals [59, 60], as made in radio frequency (RF) domain, is one of the first possibilities of contrasting this problem. In contrast, the use of cryptographic keys in acoustic messages is not easily adaptable to the underwater environment. Therefore, an interesting concept is that of physical security [61], where the signal itself can be manipulated to generate confidential keys according to channel variability [62, 63]. A recent attempt to resolve the security issue at different levels of the network is addressed by Dini et al. [64], where the authors suggest a security framework specifically designed for underwater acoustic networks. This idea was implemented into the network deployed at sea in the context of the European project UAN [65]. Finally, it is important to point out that a cooperative robotics network can use its own versatility and adaptability to respond to cyber threats as well [4••, 66].

In this context, robotic surveillance systems are also required to be indefinitely persistent, regardless of the sea state, while maintaining a high level of effectiveness. Such a long-lasting use requires the development of innovative launch and recovery systems (LARS) systems and docking stations able to harvest energy 24 h a day at sea. All this involves a number of engineering problems that need to be tackled with special attention in the underwater field. The simple deployment and recovery operations of the surveillance vehicle team are often difficult and generally require a ship with adequate lifting and dynamic positioning (DP) systems. Solid, convincing and scalable engineering solutions for autonomous deployment, docking and recharging have to be pursued in order to ensure sufficient persistence and to relieve the workload of the support vessels [47, 67]. To date, one of the key challenges that researchers are facing in improving the persistence is the precise positioning of underwater vehicles, where the constraint of the drift of error over time becomes a crucial barrier to long-term missions as surveillance [68]. Finally, lowering vehicle energy consumption by improving the efficiency of power supply and propulsion systems [69] is essential if the duration of the mission would be extended from hours to days or even months in the case of gliders [70,71,72]. A complementary approach is to recharge the internal batteries of the vehicle during the mission at offshore docking stations [73,74,75]. A more flexible idea, almost unexplored so far, would be to equip the vehicle with a portable device capable of harvesting energy from the surrounding environment. For this reason, wave motion is especially appropriate because, theoretically, it is not limited by time and space, although the process of energy conversion is proving to be very difficult [76]. This last line of research includes the Wave-powered Autonomous Vehicle for marine Exploration (WAVE) project [77, 78] whose final objective was to study, develop and test at sea a new wave energy harvesting and low-energy propulsion system to be integrated into a generic, modular, torpedo-shaped AUV.

In the SEALab framework [79•], the joint laboratory between the Naval Support and Experimentation Centre (CSSN) and the Interuniversity Centre on Integrated System for the Marine Environment (ISME), the authors outlined the recent advances on the use of maritime unmanned systems for military and dual-use applications.

Conclusions

This paper provides a picture of the current state of the art about the involvement of robotic technologies within the underwater surveillance field of application. The attention is mainly focused on the impact that the use of unmanned vehicles had within this scope. These robots can host onboard a wide variety of sensors making them suitable to various typologies of operations; at the same time, the integration with the more traditional manned solutions and, in some cases, their partial substitution makes the unmanned vehicles a solution able to lower the overall operational costs. In addition to the versatility and the cost-effectiveness of the robot involvement, a great advantage of these technologies is their impact on human safety that drastically increases as the operators are shifted far from risky areas. These intrinsic advantages of robots make them gain more and more space in the operations typical of underwater surveillance.

The paper provides a review of the recent examples, documented within the state of the art, of unmanned vehicles involved in two specific classes of operations typical of the underwater surveillance (Robots for seabed exploration, Section 2; Intrusion detection Robots, Section 3). The reader is critically guided through the recent bibliography on the topic, receiving references to the main recent advancements in the field and to sources about mission and stakeholders’ requirements (port authorities, coastal guards, navies).

The discussion of the current picture on the theme made possible to trace the road ahead towards greater and greater involvement of robots in the underwater surveillance field of application. These leading/primary research frontiers about underwater surveillance are widely discussed in the last section with references to the related bibliography. The main identified challenges include the autonomy enhancement and team/swarm mission capabilities to be achieved by the improvement of interoperability among robotic vehicles and by enabling communication networking capabilities, a non-trivial task considering the severe limitations in bandwidth and latency of acoustic underwater messaging.