1 Introduction

Soft robots usually have compliant bodies, composed of elastomer material to perform tasks in dynamic and unstructured environments [1]. Such compliance also extends the functionality of robots toward human interaction and biomedical applications [2, 3]. Similar to living creatures, soft robots also need sensors to retrieve meaningful information from the environment (exteroception) and monitor the internal states (proprioception) for corresponding actions [4]. The sensing signal from the outside is essential to achieve closed-loop control for higher control accuracy [5]. Unlike the traditional rigid robots, soft robots have an infinite degree of freedom (DoF), plus the system typically showed high nonlinearity and hysteresis [6, 7]. These characteristics pose significant challenges to soft robotic sensing and control [8, 9]. Embedded soft sensors are viable solutions, as they are deformable [1, 10] and show less mechanical mismatch when applied to soft robotics.

Among the common materials for flexible sensors (such as carbon nanotube (CNT), MXene, and reduced graphene oxide (rGO)), hydrogels are especially promising to address the challenges. They are three-dimensional cross-linked hydrophilic polymer networks containing water [11]. Hence, the mechanical properties can be tuned [12] to be highly stretchable, making it especially suitable to apply for soft robot sensing and actuation. Since hydrogels serve as the building block for life [13], they are ideal for making electronic skins by mimicking biological skins. The hydrogel mobile ions serve as the ionic conductors, endowing it with tuned conductivity. They are also transparent for light-based sensing. With the incorporated ionic pendant groups or salt, hydrogels can respond to different stimuli by interacting with polymer networks [14] to serve as multifunctional sensors. They are already applied to detect strain [5], pressure [15], and temperature [16]. Further improvement of the hydrogel sensors may boost the sensing capability to rival biological organisms [17, 18].

After obtaining the sensing signal from the hydrogel, it is hard to form the soft robot’s state representations [19], as the governing equation is almost impractical to represent. Machine learning (ML) is a promising approach to deal with the problem, since it is data-driven and can empirically approximate the unknown model [20, 21]. Therefore, it can estimate the behaviors of the soft systems without explicitly discovering the underlying dynamics. The process of ML decoding the signals helps to reconstruct the original physical phenomenon [22, 23]. In addition, machine learning can help soft robots to perform more advanced tasks, such as manipulation [24], shape detection [23], touch recognition [25, 26], state estimation [6], and multi-modal fusion. These capabilities may improve soft robots’ proprioception, enabling reliably closed-loopcontrol [24] and responses to external stimuli [28] (Fig. 1).

Fig. 1
figure 1

The relationship among soft robot, hydrogel sensors, and machine learning

Full size image

Another nascent field exerting a significant impact to soft robots is simulation with ML [29]. The point-based methods can be coupled with deep learning to provide solutions for soft robots’ simulation and virtual data processing. It also enabled virtual predictions of the soft robot with optimized designs and controllers [15]. Therefore, the overlap among soft robots, hydrogel sensors, and machine learning is increasing dramatically [30], and it is worthy of more review papers to dive into the interdisciplinary field.

This review is organized by first outlining hydrogel engineering techniques, since they are essential for soft robots applications. Then, we systematically introduce how hydrogel sensors and actuators contribute to the soft robot’s performance. Next, we dive into how machine learning can be leveraged for soft robots’ sensing signal processing and optimized design by simulation. Last, future opportunities and directions are enlightened for upcoming researches.

2 Hydrogels for Soft Robots

2.1 Hydrogel Engineering

Soft robots may be subject to large strains, so the hydrogel-based devices need to be stretchable. To cater for this requirement, chemical crosslinking of hydrogels by covalent bonds or physical entanglement are common synthesizing methods. Yet the material may suffer from slow stimuli response and poor self-healing. To solve the problem, approaches like double network [31, 32], hybrid-crosslinking [33, 34], fiber reinforcement [35], nanoparticle [36,37,38], and slide-ring cross-linker [39] were developed. These methods are closely correlated with the hydrogel’s microstructure. An example of a stretchable hydrogel’s molecular structure is shown in Fig. 2a.

Fig. 2
figure 2

a Schematic illustration of a hydrogel’s molecu-lar structure b Photographs of the stretchable hydrogel during a stretch-release process. c Micrographs of the self-healing process adapted from [40] by Zhouyue Lei & Peiyi Wu, 2019, Nature communication, Creative Commons license

Full size image

Double network hydrogel’s two networks are cross-linked by covalent bonds. One network has short chains, and the other has long chains. When the ionic gel is stretched, the short-chain network ruptures and dissipates energy to make the gel more stretchable Fig. 2b while tolerating defects and notches. The significant enhancement can be credited to the synergy of two mechanisms: crack bridging by the network of covalent cross-links and hysteresis by unzipping the network of ionic cross-links. Note that the sensing device fabricated with this method might show hysteresis, which needs engineering approaches to tailor it [41, 42].

Another critical requirement is the device’ robustness and reliability, which can be achieved by self-healing Fig. 2c and water retention. The healing mechanisms can be categorized as extrinsic and intrinsic behaviors. Extrinsic healing depends on the small capsules of healing agent [43] or vascular networks [44] to form new polymer chains that connect the fractured parts. Intrinsic mechanisms can heal more times by the reorganization of dynamic covalent bonds or through noncovalent interactions [45], like hydrogen bonding, π−π stacking, metal–ligand interaction, and electrostatic interactions [16, 46, 47]. Water retention can be achieved by combining hydrogel with elastomer [48] or adding gelatin-glycerol. The prior method can use sticking or coating to form, and adding gelatin-glycerol can be achieved by 3D printing [45, 46, 51].

2.2 Hydrogel Sensors

Unlike traditional electronic systems that transmit electricity with electrons, hydrogels use ions [52] to conduct electricity, making them perfect for sensing and signal conduction [53]. The hydrogel sensors can be divided based on the energy conversion approaches as follows.

2.2.1 Mechanical Energy to Electrical/Optical Energy

When the hydrogel sensors are subjected to mechanical deformations or applied forces, they produce a change in the electrical signal. Strain sensors, pressure or tactile sensor [54, 55], acoustic sensors, and touchpad [56] can all be summarized into this category. The electrical-based sensing mechanism can be capacitive, resistive, piezoionic, and triboelectric.

The capacitive sensors’ configuration is generally designed as the dielectric elastomer sandwiched between the hydrogel Fig. 3a. When subjected to pressure or twisted by external torque, the distance between the hydrogel changes, reflecting a capacitance change. They are fast responsive with high linearity [57], yet they are susceptible to environmental contamination and conductive objects [58].

Fig. 3
figure 3

a The schematic illustration and a photograph of a capacitive sensory system using gel and elastomer. b The structure of a typical resistive sensor. c The capacitive and resistive response of the sensory system upon changes of temperature and strain. The figure is adapted from [40] by Z. Lei & P. Wu, 2019, Nature communication, Creative Com-mons license. d The embedded sensors on the top of the soft finger are used to estimate the soft actuator’s twisting [59]

Full size image

Meanwhile, the resistive type sensor responds to changes in cross-section and length to sense the resistance change Fig. 3b. With strain-sensitive additives, the sensitivity can be even higher. Current issues associated with the resistive type are temperature-related drifting and poor long-term stability. On the other hand, making use of these temperature coupling effects, the strain sensors can be applied for multi-modal sensing Fig. 3c, d.

Here we show an example of hydrogel applied on the soft robot’s sensing using resistance measurement Fig. 4a, b. Highly sensitive strain and pressure sensors are integrated as the artificial electronic skin for soft robots by 3D printing Fig. 4c, providing proprioceptive or haptic feedback. To fully utilize the sensors, researchers integrate redundant resistive strain and pressure sensors at different locations of the soft robot, so that the sensor network can obtain more information and monitor the task manipulation in a time series.

Fig. 4
figure 4

a Images of the interaction sequence between a ball and a soft robotic gripper. b ΔR of each gel sensor as a function of time is plotted during the interaction sequence. c Schematic illustrations of the actuator with somatosensation enabled by ionogel. The figures are adapted from [67], by R. Truby et al. (Reproduced with permission). d Fast bending actuation of the hydraulic hydrogel actuator. The figure is adapted from [80], by H. Yuk et al. 2019, Nature communication, Creative Commons license

Full size image

Alternatively, utilizing the transparent characteristic of ionic gels, the optic signal can be captured when subjected to mechanical stimuli. Using composite of ZnS–silicone gel–silicone elastomer as the material system [57], ZnS particles impart a soft robot with the ability to vary its color and sense both external touch and shape changes. Alternatively, a tactile sensor can use a camera to track the sensor surface to measure both the magnitude and direction of an applied force. These vision-based tactile sensors can measure the 2D texture and 3D topography of the contact surface, utilizing elastomeric gel as the sensing surface and a camera to capture the contact deformation from changes in light. This type of 2D images are especially suitable for deep learning processing, leading to higher-level tasks.

The latest studies on hydrogel’s piezoionic effect is promising to create ionic skin with sensing ability similar to biological skin. The anions and cations in hydrogels have different mobility. So when the material is pressed, it causes an ionic gradient that generates voltage [60]. The self-powered method can be applied for neuromodulation with living animals and monitoring different motions [61,62,63]. Meanwhile, it can also sense multi-modal signals using the thermoelectric effect [64].

Another recently emerged type of triboelectric nanogenerator is made with hydrogel to perform tactile sensing with high performance [54]. Typically, the hydrogel part needs to be coated with elastomer, and it generates electricity when separated from the dielectric layer. In comparison, the voltage generated by triboelectric effect can be very high, showing high sensitivity to dynamic pressures. It is even possible to detect precontact event by optimizing elastomeric electret, which greatly expand the functionalities of soft robot [55]. Some recent studies of hydrogel sensors applied to soft robots are listed in Table 1.

Table 1 List of hydrogel sensors applied on soft robot research
Full size table

2.2.2 Thermal Energy to Electrical Energy

Thermal sensing is also essential for soft robots to identify the object and avoid hazardous environments [68]. The hydrogel thermal sensor can convert the temperature stimuli into electrical or optical responses. The electrical-based sensing can be divided into three categories, resistive, capacitive, and thermoelectric type.

The resistive thermal sensor’s mechanism lies in the temperature dependent ions migration rate [40]. Due to the material’s high sensitivity, they were seen applied as the thermal transducer [71]. To characterize the thermal sensitivity of a resistor, the temperature coefficient of resistance (TCR) is often used. It describes the percentage change of resistance per Celsius degree. The double network hydrogel exhibited a TCR of 2.6%/℃ [31], which is already more sensitive than some semiconductor thermistors. By introducing ethylene glycol/glycerol, extremely high TCR values of 19.6%/℃ [73] and 24%/℃ [73] are reached.

The capacitive type utilizes the hydrogel swelling and shrinking property to convert the thermal stimuli into a change of capacitance. When the thermal responsive hydrogel is subjected to temperature changes, it will show a volumetric transition as a result of the interaction between the polymer network and the solvent [74].

Recent emerged hydrogel-based thermoelectric material showed surprisingly high seeback coefficient. The work uses alkali salts and an iron-based redox to generate 17.0 millivolts per degree Kelvin, which is two orders higher than traditional thermoelectric materials [66]. Later, the hydrogel made by synergistic coordination and hydration interactions [77] showed thermal sensitivity of − 37.61 millivolts per kelvin. These encouraging findings indicate hydrogel can play a significant role in soft robots’ self-powered thermal sensing.

2.3 Hydrogel Actuators

Aside from sensing applications in soft robotics, hydrogels can be applied as actuators to swell or shrink [77]. The swelling degree of the hydrogel is determined by the free energy of expansion and the entropy of mixing. They could be affected strongly by electricity, temperature, pH, light. These condition changes could act as the stimuli for actuation [78].

The most frequently seen actuator is the dielectric elastomer actuator (DEA). It comprises a dielectric elastomer layer, sandwiched between two ionically conductive hydrogel layers. When a high voltage is applied between the hydrogels, ions of opposite charges accumulate along the hydrogel/elastomer interface. This induces Maxwell stress between the hydrogels, resulting in thickness contraction and area expansion. The development of hydrogel-based actuator showed promising applications in various complex environments. A pressure-resilient soft robot made of DE and hydrogel swimming at extreme ocean depths has been studied recently [79]. To make soft actuators with high-speed and high-force, hydraulic actuation of hydrogels has been explored Fig. 4d. This soft robot can be sonically camouflaged in water [80], making it very promising for future underwater exploration.

3 Machine Learning for Soft Robot

Some recent research on machine learning techniques applied to soft robots are listed in Table 2. To better illustrate how the field is progressing, we use several key features, such as sensor used, data type, multi-modal or not, ML technique, and its applications, to categorize these works.

Table 2 Recent soft robot studies with machine learning for data processing
Full size table

3.1 Machine Learning for Sensing

Machine learning-based method is promising for both soft robot proprioception and objects recognition when resistive or capacitive readings are available [92, 93]. The object identification was seen frequently based on the finger curvature data using machine learning algorithms [81]. The techniques can be convolutional neural networks (CNN) and recurrent neural networks (RNN). RNN deals time series data more. It can be used for inverse mapping problems and flexible strain sensors modeling. Researchers used LSTM (Long short term memory) to estimate the contact force in the soft robots [6], and the approach can model the kinematics of a soft actuator in real-time while being robust to sensors’ nonlinearity and drift. Further optimization was seen using transfer learning to improve the performance [83]. Some studies also utilize CNNs for time series feature extraction, yet CNNs appeared more in problems requiring two or three dimensions data [94]. Capacitive arrays can provide sensing data in two dimensions. By feeding the data into CNN or feed-forward neural networks (NN), they can classify and localize touch events on the interface or capture surface deformations in real-time [26, 95].

Other types of sensing modality can be combined using ML techniques to boost the sensing capability [29]. When thermal conductivity, contact pressure, object temperature and environment temperature are fed to the multiple layer perceptrons (MLPs) simultaneously, machine learning can improve the object recognition. It discerns shapes, sizes, and materials in a diverse set of objects with improved accuracy, comparing to single modal inputs [96]. Pressure sensing has been combined with other modality using machine learning to expand the sensing capability. Using machine learning techniques, such as support vector machine (SVM), the soft robot can detect obstacles and recognize the gestures [25].

Vision-based proprioception and tactile sensing can be obtained simultaneously using embedded cameras or depth sensors by extracting high-resolution information from the deformation of the soft robots Fig. 5a, b. Training the data with CNN can lead to object detection, slip detection [87, 97], robot’s shape or angle proprioception [28, 86, 98], and interaction prediction [90]. For example, in research for soft robot 3D shape proprioception, the system uses a CNN to encode the input images from the internal cameras into latent representations, and then train a decoder neural network to reconstruct the 3D shapes of the robot [85].

Fig. 5
figure 5

a The basic principle of the Gelsight b An example pattern of printed markers on the Gelsight. The figures are adapted from [101], by W. Yuan et al. (Creative Commons license) c vision to touch prediction model. The figure is adapted from [88], by Y. Li et al.

Full size image

Combining multisensory information and transferring multi-modal [22] representations across tasks was also explored in recent studies. To fuse the sensor data and perform prediction, a feature representation of the visual and haptic data can form compact feature vectors. Then, decoders are used to predict self-supervised objectives [99]. In addition, deep models can incorporate raw inputs from both vision and touch, by concatenating the outputs of the camera images, GelSight images, and the action network. The output was applied to predict the grasp success probability. In this way, the soft gripper can grasp a wide range of unseen objects with a higher success rate and lower force [100]. Cross-modal prediction can also be made possible with deep learning. With visual and tactile sensors, researchers can collect a dataset of vision and tactile image sequences. Then, a conditional model that incorporates the scale and location information of the touch can be transformed to vision. The deep learning model requires a generator, with two CNN-based encoders and one decoder. In this way, it uses vision sequence and touch reference to predict the touch results [88] Fig. 5c.

3.2 Machine Learning for Simulation and Control

Simulation can aid robot characteristics prediction and optimal design. Prior attempts using finite element analysis (FEA) were computationally expensive [102]. Recently developed physics engines are combined with deep learning to optimize soft robots Fig. 6a. Such gradient-based optimization methods can be more computationally efficient [103]. Besides, by simulating virtual robots using data-driven models, we can simultaneously optimize multiple objectives, such as geometry Fig. 6b, controller models, [104] and physical system properties for system identification.

Fig. 6
figure 6

a Co-optimization algorithm pipeline for a biped soft walker. The figure is adapted from [25]. b Computational design pipeline for underwater soft swimmer’s geometric and control design parameters. The figure is adapted from [96] (Creative common licence)

Full size image

When using deep learning for soft robot control, the regression-based controller can be designed in this way. It takes the input state vector z, which includes the target position, the center of mass position, and the velocity of the soft component. During optimization, the algorithm performs gradient descent on variables W and b, and a = tanh (Wz + b) is the actuation-generated from the controller. Using the deep learning-based controller, a soft 2D walker optimized using gradient descent can find the gaits to achieve maximum distance [95].

Since the soft robot has infinite numbers of DoF, modeling the compliant robot is complex, and reinforcement learning can be applied to address the complexity [105, 106]. The closed-loop control can also be achieved in a simulation environment with reinforcement learning. In this way, the model-free method can deal with the soft robot’s challenging deformation scenarios [107].

When the hybrid particle-grid-based simulation is combined with deep variational convolutional autoencoder architectures, it can capture salient features of robot dynamics for improved state estimation, control, and design [108]. Meanwhile, with the differentiable physical engine that simulates elastic deformation, it is possible to conduct supervised learning on multiple soft-body manipulation tasks with different configurations [109]. Moreover, based on simulation data, the feature extraction network can learn the representation of sparsely located sensors for optimized object grasping and proprioception [84, 110].

4 Future Challenges and Opportunities

In summary, there has been tremendous progress in the performance and versatility of soft robots in recent years, leveraging advanced machine learning techniques and gel type materials. To realize the full potential, fundamental developments in materials engineering, smart structures, sensing mechanisms, and data processing algorithms are required to boost the progress of soft robotics [4, 111].

We believe that ionic gel-based sensing, especially multi-modal stimuli sensing, will play a critical role in the sophistication of soft robotics. However, several things need to consider. First, how to optimize the wiring for large area haptic sensor arrays? This question hampers soft robots’ real-world applications. Also, resistive tactile sensing using hydrogel can lead to unwanted drift, as the hydrogel is sensitive to temperature changes, which is hard to find real-time solutions to avoid it.

Using machine learning to aid in sensing and control is a promising way, as it could impact the space of what is possible with intelligent robotics systems. Yet, there remained challenging tasks to solve. For instance, current 3D shape proception is still not integrated with feedback control to enable controlled shape change. Also, humans fuse different sensing modalities using maximum likelihood estimation and the Bayesian method, yet we rarely witness studies applying these principles on soft robots for multi-modal fusion due to engineering difficulties. Moreover, deep learning coupled with the physics engine will play a more significant role in the future to optimize the soft robots in multiple ways simultaneously. However, this field is still in its infancy. Future development is expected when more interdisicplinary collaborations are available.

In the short term, the field is focusing on soft robots’ sensory actuation, techniques for processing the sensor information, and feedback control. The longer-term goal lies in future exploratory robots that can navigate the unpredictable natural world. We expect to see a trend toward more complex hybrid systems dealing with various robotic tasks. In a more realistic environment, the sources of information are more diverse, so it requires more sophisticated deep learning models to fuse the multi-modal signals and perform high-level tasks. This demands the algorithms to derive efficient representations of the environment from the high-dimensional sensor inputs and use them to generalize the past experience to new situations. Humans and other animals seem to solve problems through a harmonious combination of reinforcement learning and hierarchical sensory processing systems [112]. Therefore, deep reinforcement learning is also a promising pathway for the soft robot to deal with advanced tasks [113, 114], such as decision-making and policy learning. In that way, the soft robot can perform multiple steps of action, solving more complicated tasks.