A Survey of Robotic Harvesting Systems and Enabling Technologies

This paper presents a comprehensive review of ground agricultural robotic systems and applications with special focus on harvesting that span research and commercial products and results, as well as their enabling technologies. The majority of literature concerns the development of crop detection, field navigation via vision and their related challenges. Health monitoring, yield estimation, water status inspection, seed planting and weed removal are frequently encountered tasks. Regarding robotic harvesting, apples, strawberries, tomatoes and sweet peppers are mainly the crops considered in publications, research projects and commercial products. The reported harvesting agricultural robotic solutions, typically consist of a mobile platform, a single robotic arm/manipulator and various navigation/vision systems. This paper reviews reported development of specific functionalities and hardware, typically required by an operating agricultural robot harvester; they include (a) vision systems, (b) motion planning/navigation methodologies (for the robotic platform and/or arm), (c) Human-Robot-Interaction (HRI) strategies with 3D visualization, (d) system operation planning & grasping strategies and (e) robotic end-effector/gripper design. Clearly, automated agriculture and specifically autonomous harvesting via robotic systems is a research area that remains wide open, offering several challenges where new contributions can be made.


Introduction
The growing demand for food supply that derives from the continuously increasing population 1 has made agricultural productivity growth an important priority. Labor availability pressure driven by demographics of an aging population, increasing urbanization, climate change and land degradation, as well as certain limitations regarding the arable land availability push forward slowly but steadily, the use of advanced agricultural technologies. Incorporating such technologies into agricultural production benefits the overall productivity and in turn supports the economic development and growth [1]. Additionally, automation in agriculture is bound to help improve the difficult work 1 https://population.un.org/wpp/Graphs/Probabilistic/POP/TOT/900 -Last accessed: 22-11-2022 Leonidas Droukas ldroukas@ece.auth.gr Extended author information available on the last page of the article.
conditions of farmers and agricultural workers that are generally linked to various musculoskeletal disorders [2][3][4]. Application of robotic solutions regarding crop monitoring and harvesting is considered to have significant beneficial effects on production profits [5], enabling faster and easier automated harvest and increasing crop quality and yield. Thus, the development of robotic technologies and their application in agriculture is becoming a growing topic of interest and consideration [6], with an increasing amount of research work being noticed in the last decades. This falls under the general umbrella of a growing trend in agriculture, which is termed precision agriculture or agriculture with reduced carbon footprint.
Several reviews may be found in the literature, focusing on agricultural robots [7][8][9][10]. Standardisation of terms and characteristics and system performance measures are utilized to evaluate these robotic systems for various field tasks and operations, e.g. transplanting/seeding, pruning/thinning, weed control and disease monitoring [11][12][13]. Focused research reviews have been conducted regarding technologies, dealing with specific issues in automated agriculture, such as the management and analysis of the large data sets that most robotic systems require during their operation [14] or the optimized sensing and detection required for visual-based guidance of the applied robotic agricultural systems [15]. Robotic applications that target specific crops have also been a research goal, including cotton [16], strawberries [17] or the general arable farming, e.g. wheat and rice [18]. A brief review concerning various apple harvesting platforms is presented in [5] that additionally develops a software tool for their economic evaluation, while several literature studies on agricultural robotic systems are investigated including the AgROS emulation tool [19], classifying them with regard to their analysis approach and farming operations.
In addition to UGVs i.e. Unmanned Ground Vehicles (robots), drones/Unmanned Aerial Vehicles (UAVs) have also been increasingly considered in agriculture during the last years [20]. Given their ability to easily cover large distances over the fields, aerial robots are widely utilized with respect to several agricultural tasks such as crop monitoring/mapping, pesticide/fertilizer spraying, seed sowing and growth assessment [21][22][23]. Combining the survey capabilities of UAVs with the targeted intervention ability of UGVs, multi-robot systems are gaining great attention as well [24][25][26]. Various works can be found in literature regarding the coordination of cooperative ground and aerial robots towards e.g. selective spraying [27], soil and biota sampling [28] and monitoring/inspection tasks [29].
Given the great amount of works corresponding to automated/precision agriculture, in this work we focus our investigation on ground agricultural robotic systems and applications. The presented review covers not only publications found in literature, but research projects and commercial products in the market as well, considering robotic solutions and their enabling technologies with a special focus on harvesting. Section 2 presents an overview of complete harvesting robotic systems for various types of crops. Section 3 investigates a variety of technologies typically required by an operating robot harvester including vision systems, navigation methodologies, HRI strategies and more. Sections 4 and 5 present past and existing research projects and commercial products respectively, followed by the Conclusion Section.

Complete Harvesting Robotic Systems
Robotic automated crop harvesting e.g. of fruits or vegetables has a high impact on agricultural productivity with the task being investigated and considered even from early 1960's [30]. There are two basic concepts regarding automated harvesting by robots: bulk and selective. Bulk concept involves the harvesting of all fruits without exceptions, usually considering methodologies such as tree trunk or limb shaking [31][32][33]. However, bulk methods entail the danger of harming the crops. With the development of new technologies and the possibilities they enable, the selective concept is mostly adopted in the last few years. In the selective harvesting of crops, the robotic system firstly decides which are the harvest targets (e.g. ripe fruits identified via a sensory/vision system) and then harvests them. This task may typically involve scanning the whole crop in an orchard or a greenhouse or a part of it, acknowledging and locating targets, cutting/picking them and placing them in a storage unit (e.g. a crate). In the existing literature, there is a variety of integrated robotic solutions that have been proposed towards this end. Representative robotic systems usually consist of: (a) a moving platform/vehicle upon which a robotic manipulator resides, responsible for approaching target fruits/vegetables, grasping and cutting them and subsequently placing them on a crate; (b) a vision system for crop scanning, target identification, detection and localization; (c) a specifically designed robot end-effector to best facilitate target grasping and collection.
Several harvesting robotic systems for apples may be found in the literature. Since apples have a fairly standard circular shape and a hard nature, they are generally easy to be harvested without significant damage to the fruit. Some of the fastest developed robotic systems report harvest times of 6 seconds [34]-( Fig. 1(a)) or 8-10 seconds [35]-( Fig. 1(b)) per fruit with a success rate of 80% or higher. Similar performance may be found in [36] where harvest is achieved in an average time of 7 seconds with a success rate of 90%. A slower behavior regarding harvesting time of 15 or 16 seconds may be found in other harvesters, like the fruit harvester in [37]-( Fig. 1(c)) that was tested on apples or the apple harvester in [38]-( Fig. 1(d)) where the importance of real-time obstacle avoidance that the harvesting robots should possess is underlined, due to the obvious complexity and unknown nature of their working environment.
Various robotic solutions may be found in the last decade for the case of strawberries, reporting good performance [41,42]-( Fig. 1(f)), [43] and a success rate greater than 75% for isolated strawberries with an average harvest time ranging from 6 to 10 seconds per fruit; additional to harvesting operations, such as the packing of the harvested fruits, have also been considered [44,45]-( Fig. 1(g)).
The difference of color between the target fruit and its environment is also exploited in the case of cherries making them relatively easy for detection [46]-( Fig. 1(h)). In particular, a cherry-harvesting robot was constructed with a 4 DoF arm, integrated with a 3-D vision sensor and a special end-effector that harvested the fruit with its peduncle, to locate the fruits and plan the trajectory of the end-effector avoiding obstacles [46]. Researchers have also considered the harvest of heavier fruits like melons [47,48] or watermelons in the case of "STORK" robot [49]-( Fig. 1(i)).
In order to speed up the robot's harvesting cycle time, multi-arm harvesting operating simultaneously has been explored in [50] and in [51]. The work in [50] concerns the melon field case, where the arm-melon allocation problem is addressed. A similar approach may be found in the fruit-harvesting robot Agribot [51], where two robotic arms work in parallel, harvesting the targets that are selected and marked by a human via a point laser range-finder.
The importance of developing and adapting new high productivity cultivation systems is discussed in the literature, modifying the crop and its envelopment to better fit for robotic harvesting. In particular, the high-wire system is considered for the cucumber case [57]. This issue is also raised and investigated in [58] in the SWEEPER 2 project. Such specifically designed cultivation systems can facilitate even further the automation in the agriculture, making robotic harvesting easier and faster [59], e.g. the V-trellis fruiting wall architecture for apples [34], enhancing target fruit access and presenting fewer interfering obstacles.
In addition to fruits, vegetables have also been considered as harvest targets by automated harvesting machines. An autonomous robot for cucumber harvesting is proposed in [57]-( Fig. 1(n)), reporting a success harvesting rate of 80% and a cycle time of 45 seconds for each harvest target. In the eggplant case [60]-( Fig. 1(o)), 62.5% success rate and an average harvest time of 64.1 seconds are reported. Works regarding radicchio [61]-( Fig. 1(p)) and asparagus [62] may also been found in the literature. Relatively low success rates in crops detachment are reported in [63]-( Fig. 1(q)), where the presented autonomous harvester for crops with peduncles (either vegetables or fruits) is tested on both plastic and real crop cases, exhibiting a 67% and a 52% success rate respectively. A research interest has been shown for sweet peppers as well, in [64,65]-( Fig. 1(r)) with the "Harvey" robot and in [66]. Specifically, [66] is part of the EU project SWEEPER that targeted towards the development and testing of a practical robotic harvesting solution for sweet peppers in real-world conditions, reporting an average harvest time of 24 seconds per pepper.
Summarizing, as can be seen in Table 1, it becomes clear that the majority of harvesting robotic systems involve crops with shape, size, rigidity and color that facilitates automated harvesting. The lack of any reported harvesting solutions for grapes which is a high-value crop indicate the existence of specific challenges in this case. Nevertheless, various research works may be found for the case of grapevines.
As part of the PIC4SeR 3 mission for intelligent vineyard healthcare, the Agri.q UGV is developed in [67][68][69], with its operation extended in other types of orchards as well. Overcoming terrain irregularities or slopes and providing various services such as crop health monitoring, collection of soil and/or plant samples or application of fertilizers, this robotic solution investigates the innovative possibility of multiple robotic systems working collaboratively for coordinated field monitoring and servicing, including a UGV and a UAV (drone). An early attempt for developing a multipurpose agricultural robot working in vineyards can be found in [70], considering various tasks such as berry thinning, spraying, bagging and harvesting. The robotic manipulator developed in [70] was also utilized in [71], where the explicit task of spraying chemicals under grapevine trellis for crop health/disease removal purposes is addressed. Disease detection and selective spraying for production quality enhancement is the research target of [72] as well, part of the EU-project CROPS 4 whose main goal was to develop, optimize and demonstrate a multipurpose agricultural robotic system [73]. Project CROPS attempted to achieve grape harvesting as well, however with no reported success. A similar application, Harvesting Apple [34][35][36][37][38][39] Harvest times / success rates: 6-16 sec / 80-90%. Methods for faster/easier harvest: e.g. pick-and-catch harvesting [39]. V-trellis farming architecture [34]. Orange [40] Early interest shown (1990's). Harvest-friendly crop shape and color. Strawberry [41][42][43][44][45] Success rates > 75%. Harvest times: 6 to 10 sec. Packing operation [44,45]. Cherry [46] Different fruit/environment color exploited to facilitate detection. Melon [47,48,50] Simultaneous multi-arm harvesting [50] to speed up harvest. Watermelon [49] Reported success rate: 66.7%; robot weighting < 300 kg picking fruits of 13 kg. Tomato [52][53][54][55][56] Dual-arm harvester [55,56] contrary to common unimanual systems. Cucumber [57]  i.e. vineyard protection, was also targeted by the GRAPE 5 project. In GRAPE, an autonomous ground robot [74] with a robotic arm was developed for health monitoring and automatic pheromone dispenser distribution for plague control. Other works in literature involve tasks like weeding i.e. removal of weeds in areas of a vineyard [75], monitoring i.e. localization and mapping of vineyard for crop monitoring [76] and pruning i.e. cutting off some of the grapevine older canes/branches while leaving others that are healthier [77]. Through utilization of two independent robots working cooperatively (harvesting and inspection robots), ongoing EU funded project BACCHUS 6 aims at contributing in the limited field of automated grape harvesting. Introducing a human-like harvesting approach via a bi-manual setup (harvesting robot) contrary to most existing solutions, while combining it with crop inspection, mapping, as well as vital data collection e.g. concerning grape health/maturity (inspection robot), BACCHUS targets on pushing forward the application of precision agriculture in a particularly challenging environment such as a vineyard (BACCHUS as well as the previously mentioned GRAPE, CROPS and SWEEPER will be described further in Section 4, where the research projects are presented in details).

Robotic Functionalities and Hardware for Automated Harvesting
Apart from the development of a complete robotic system for the automated harvesting of fruits or vegetables, one may find a significant amount of works in the literature that deal with solutions over specific challenges related to the overall harvesting task. Vision-related challenges regarding harvest targets detection and their maturity and health evaluation, navigation of robotic systems in fields and orchards, motion and grasp planning of the robotic system's manipulator for successful harvesting, design of end-effectors with specific requirements depending on the harvest crops as well as operation planning and human robot interaction (HRI) systems for the agricultural domain are some of the problems addressed in the literature. Figure 2 depicts how the referenced publications are distributed with respect to their targeted functionality, indicating that the majority of them address crop detection and evaluation.

Sensor-based Detection and Evaluation of Harvest Targets
The detection of the harvest targets as well as the crops' health and maturity evaluation are of high importance for the farmers concerning both estimating the yield quantity and quality, with evident impacts on the financial earnings and managerial issues (e.g. arrangements for crop shipment or storage), and the actual application of robotic harvesting methods. To this end, robotic systems with various visual sensors/cameras (e.g. binocular vision, laser vision, kinect camera, multispectral camera etc.) placed on the robotic platforms themselves and/or mounted on the robotic manipulator's end-effector (eye-in-hand) are proposed, while image processing methodologies are employed to transform the raw imagery data to valuable knowledge [78]. The main tasks addressed in the cropdetection literature refer to the fruit recognition, which is the key issue in the vision-guided autonomous fruit harvesting and yield estimation systems, the crop's maturity/health evaluation, which is critical on the crop's suitability for harvesting decision-making, and the environment modelling and surveying, which provides useful input for the autonomous agricultural applications.

Vision-based Crop Detection
Over the last years, a great variety of research work has been conducted regarding vision-based detection methods for utilization in various automated agriculture applications [79]. Adaptive thresholding algorithms [80], methods based on texture, color and shape cues [81][82][83], combination of RGB and depth information [84], reinforcement-learning approaches [85] and deep convolutional neural networks [86,87] are used for image segmentation into background and foreground in order to detect target fruits. Additionally, to properly position the robot with respect to the target, localization of the harvest targets may also include estimation of their orientation [88][89][90] leading to the robotic end-effector approaching its target in a way that is better suited for grasping/cutting the peduncle. Harvest targeted fruit detection is a task that faces multiple challenges due the dynamic and unstructured nature of both the crops and the outdoors environment. Therefore, to cope with color and texture-based sensitivities to the variety of illumination conditions, deep learning approaches [91] and RGB-D data methods are proposed [92,93]. Moreover, occlusions that often occur due to leaves of branches affect directly the harvest-target detection success. To address this problem image analysis including per-pixel segmentation and region detection is used in [94], while a multi-class approach that splits citrus samples into different cases (normal, branch occlusion, leaf occlusion, slight occlusion, overlapping and main brunch) is applied in [95]. Furthermore, the authors of [96] propose an active vision approach that utilizes 9 eye-in-hand RGB cameras to solve the "next best view problem" for sweet peppers occluded by leaves, while [43] combines vision with specific movements that the robotic manipulator's end-effector performs in order to push away any surrounding obstacles that occlude the harvest target. Apart from the difficulties occurring from the environment, there are challenges created by the fruit's nature. In this scope, the challenging color of certain fruits, such as green grapes, is handled using different color spaces depending on the kind of crops and considering additional features during the image process analysis for the detection/localization task ( [97,98], parts of SWEEPER and CROPS projects respectively). Moreover, vision issues such as the lack of contrasting features on smooth roundsurfaced fruits that otherwise could help in their detection is dealt by utilization of the gradual variation of intensity and the gradient orientation upon the fruit's surface [99].
A group of works on agricultural object detection utilizes deep neural architectures for semantically segmenting the scene in conjunction with methods for clustering the resulting segmentation map. The authors of [100] compute the semantic segmentation of apple orchard RGB images using multi-scale Multi-Layered Perceptrons (MLPs) and Convolutional Neural Networks (CNNs) that are extended by incorporating metadata, such as sun position, while on top of the segmentation the Watershed and the Circular Hough Transform algorithms are utilized to perform fruit detection. The authors of [101] detect guava using a Fully Convolutional Network for semantically segmenting RGB images and then extract clusters that correspond to individual fruits. In [102], Unet is utilized to semantically segment RGB images into soil/vegetation classes, then blob extraction is applied, and finally the results are classified into weed/crop using a CNN classifier with VGG16 backbone. Moreover, in [103] a Unet followed by connected component analysis, a Faster-RCNN with FPN and a focal loss term, and a Gaussian Mixture Model are compared for fruit detection in apple orchards in order to achieve 3D reconstruction of the trees. A fruit-counting algorithm is presented in [104] that incorporates an object detection step with FCN based semantic segmentation and a contour detection step. Furthermore, the authors of [105] utilize a probabilistic image segmentation method to segment RGB fruit images in foreground/background, mask the corresponding depth image with the probabilistic segmentation output, and then employ a region growing method to obtain clusters from the depth data and detect fruits through an SVM classifier. Some works focus solely on the problem of semantically segmenting the captured agricultural scene. In [106], the authors employ a slidingwindow Inception-v3 classifier to segment RGB images in three classes: crop, weed, and background. Many authors utilize well-known one-stage or two-stage deep CNN detectors, using the algorithms either off-theshelf or with custom modifications, to directly detect fruits and/or plants in their input data. In this scope, the authors of [107] compare YOLOv2, YOLOv3 and Mask-RCNN in order to implement a fruit counting graph-based application, while various applications of YOLOv2 and YOLOv3 are also used in [108,109] for mango and apple detection. The authors of [110] use Mask-RCNN for blueberry detection and quantification, investigating different feature extractors/backbones for the task, while in [111], a Mask-RCNN with RGB & HSV 6-channel input is utilized to detect oranges. Moreover, Faster-RCNN is used in [112][113][114] for the detection of multiple types of fruits based on RGB and Near-Infrared images, the identification of Fuji apples on RGB-D images and the detection of 4 different wheat growth stages respectively. In [115], an automatic label generation module named Clustering-RCNN, along with the one-stage deep learning object detector LedNet are developed for the detection of apples in RGB images of orchards, while the same authors propose in [116] a DaSNet, which utilizes a Gated Feature Pyramid Network for multi-level feature fusion, ASPP for multi-scale information fusion, and a lightweight designed backbone. A strawberry detector is employed in [117] based on a modified RetinaNet network that utilizes ResNet-18 as its backbone, FPN for multi-level feature fusion and takes as input a concatenated 6-channel RGB + CIELab representation of the image, while in [118] a novel object detector for flowers is proposed, based on regions generated by the SLIC algorithm that are used from a CNN architecture to extract features that are fed into a SVM in order to detect apple flowers.

Field/Orchard Modelling and Surveying
Besides the aforementioned work that focuses on the crops, some researchers investigated issues related to the whole agricultural environment, i.e. the fields and orchards. The authors aim at modelling or surveying the environment using various visual sensors (e.g. spectral cameras, RGB, RGB-D, LiDAR and thermal imaging sensors) while it can be observed that phenotyping has attracted the interest of many authors. Phenotyping refers to the quantitative description of a plant's properties and aims at the holistic evaluation and characterization of its performance. To this end, a mobile platform for monitoring Canola plants is presented in [119]. The platform carries different sensors, cameras and other measurement equipment while it geo-tags the acquired data using GPS for future retrieval. Moreover, a fully automated robotic platform moving on rails and mounted with various sensors (chlorophyll fluorescence and thermal infrared cameras, two hyperspectral imagers and dual 3D laser scanners) is established in [120] in order to facilitate continuous and high-throughput monitoring of crop performance. An autonomous ground vehicle named "Vinobot" and a mobile observation tower named "Vinoculer" are introduced in [121] for the inspection of large field areas during day and night in order to provide high-throughput plant phenotyping. The ground vehicle collects data from individual plants, while the observation tower oversees an entire field, identifying specific plants for further inspection by the Vinobot. Furthermore, the authors of [122] address the grapevine varietal classification while avoiding the destructive techniques of classic ampelography. Particularly, an on-the-go hyperspectral imaging system is proposed that, by means of SVM and MLPs, successfully classifies leaves from different grapevine varieties.
To achieve field and orchard monitoring and analysis, image processing methods are commonly used. Therefore, semi-supervised weed classification based on CNNs is proposed in [123], while an anomaly detection system is presented in [124] that uses the features derived from a CNN classifier to detect anomalies in agricultural fields. Moreover, the authors of [125] utilize multi-view images and a probabilistic setting to infer the existence of branch structures hidden under leaves and to reconstruct them using particle flow simulation, whereas in [77], the 3D model of the whole orchard is reconstructed by means of stereo cameras. A system able to recognize the canopy volume is proposed in [126] using two LiDAR sensors aligned vertically for scanning the same targets so as to provide information for the pruning task. Furthermore, in [127], an automated method performed on hyperspectral images is developed in order to segment raspberry plants from the background using a selected spectral ratio combined with edge detection while graph theory is used to detect the continuous boundary between uninteresting plants and the area of interest.

Crop Evaluation
A vital issue during the harvesting season is the evaluation of the fruits mainly in terms of maturity estimation. In this scope, several works have explored the use of hyperspectral cameras so as to achieve non-destructive evaluation approaches. Therefore, VIS-NIR spectroscopy has been mainly used to examine point spectroscopy in laboratory environments. In [128][129][130][131], high-resolution spectrometers capture single spectral signatures from grape berries and bunches under specific illumination conditions to achieve maturity evaluation. On the other hand, there are also efforts to use hyperspectral cameras in orchards while applying methods that handle the shadows. In particular, the authors of [132] use a ground vehicle called "Shrimp" equipped with a hyperspectral camera to estimate the dry matter content (DM) of mangos while LiDAR data are used for tree segmentation so that DM predictions are projected to individual trees. The authors extend their work in [133,134] with pre-processing techniques for illumination compensation while they explore different optical filters for mango DM estimation.
Apart from the fruit maturity, authors have explored the evaluation of fruits regarding other criteria, such as checking for damages. In this scope, a modified version of YOLOv3 is used in [112] that incorporates DenseNet layers to detect apple lesions in a dataset that has been augmented by synthetic images created from CycleGAN, while in [135] YOLOv4 CSPDarknet53 method is proposed for detecting damages in sugar beet. Using VIS/NIR transmission coupled with diameter correction and a deep 1D-CNN, the early detection of freezing damage in oranges is achieved in [136]. The authors of [137] implement a per-pixel classification/semantic segmentation of hyperspectral healthy and diseased wheat data, experimenting with two different input configurations in a Deep Convolutional Recurrent Neural Network. Furthermore, the authors of [138] provide a complete and detailed crop 3D reconstruction through image data analysis to be used for phenotyping in order to further explore the crop's features such as disease resistance, breeding efficiency etc.

Detection and Evaluation of Harvest Targets Using Drones
Crop detection and evaluation functionalities by drones have shown a wide potential in precision farming [139] which, supplementary to ground robots could further boost automation in the farming domain. Mainly focusing on crop monitoring and weed/disease detection, the drone applications mostly use three types of sensors: 1) RGB, 2) multispectral and 3) hyperspectral sensors [140]. Based on RGB input, the authors of [141] monitor fields for yellow rust using U-Net deep learning networks while CNNs are used in [142] for detection of plant diseases that affect their leaves. Moreover, weed detection takes place using a Bag -of Visual -Words framework in [143] and various segmentation methods (CGNet, ENet, ERFNet) are compared in [144] towards dandelion identification. Using multispectral sensors, weed detection and crop monitoring is also explored in [145] and [146] while, due to the availability of the high number of radiometric bands, [147] extends to the extraction of non-visible field features such as early stage plant disease and soil water content. Furthermore, hyperspectral data is used in [148] to identify the most significant spectral bands and perform weed classification based on discriminant analysis as well as in [149] to evaluate maize and weed response to herbicide applications.

Motion Planning and Control for Grasping and Picking a Harvest Target
Motion planning and control of a harvesting robotic system has been addressed by various works in literature regarding either guiding the system's vehicle/platform inside the orchard (between plants) or planning of the system's robotic manipulator motion in order to approach and pick a harvest target. When guiding a robotic vehicle/platform in such a difficult environment as an orchard with many obstacles (e.g. plants, trees, branches), utilization of feedback provided by various sensors is essential. Global Positioning System (GPS), accelerometers and gyroscopes (Inertial Navigation System -INS), vision i.e. cameras, visual odometry (VO), as well as a fusion of the above feedback data [150] are often considered.
Several works address the problem of robot's navigation within fields or orchards. The development of a navigation system that can guide a field robot to travel from a farm station to a citrus grove and visit each tree autonomously with obstacle avoidance ability is the subject of [151]. GPS, IMU and the wheel encoders are integrated by an extended Kalman filter to provide accurate odometry information. Then, the SLAM algorithm combines the odometry data with the LiDAR data, enabling the robot to do path planning and navigating based on laser scans. To facilitate navigation, the robotic platform "BoniRob" [152] for crop scouting, developed by AMAZONE 7 , utilizes the FX6 laser to detect the ground from a 3D point cloud recorded by FX6. Furthermore, convolutional neural networks in combination with visual-based feedback from a camera are also proposed for crop identification, driving path planning and fitting between plant rows [153].
Concerning map generation, there are many works for indoor environments which give satisfactory results [154]. Nevertheless, there are still few proposals concerning the mapping in agricultural environments. Their goal is to decrease navigation errors [155] as well as to enable the vehicle to return to specific locations and perform tasks, such as spraying, in a suitable and precise manner, thus saving valuable resources [156]. Such example is described in [157] where a mapping strategy based on maize plants detection using a stereo camera is presented. In recent years, many works have been made to solve the problem of simultaneous localization and mapping (SLAM), providing an appealing alternative to user-built 7 http://www.amazone.de -Last accessed: 22-11-2022 maps and showing that robot operation is possible in the absence of an ad hoc localization infrastructure [158]. Cheein et al. [159] developed a precision SLAM algorithm based on extended information filter (EIF-SLAM) for agricultural environments, in their case olive groves. The most significant features in the agricultural environment are used for the optimization and pose estimation process. Moreover, the use of a SLAM algorithm, called graph-SLAM, is proposed in [160] as a means to generate costeffective local maps of forests. In particular, the 3D map is generated from laser scans mounted on a mobile platform, first by relying on laser odometry and then by improving it with robust graph optimisation after loop closures.
Handling the manipulator's motion towards the harvest target is considered a rather challenging task taking into consideration the highly unstructured environment of orchards. CROPS project [161] is a representative work that deals with most of the typical issues arising during the manipulator's motion towards the harvest target by approaching it dynamically taking into consideration possible obstacles. Collision maps are typically generated and can be also updated in real-time as the robot moves, often including 3D point cloud data [162] provided by feedback from the vision system and other sensors the overall robotic system may be equipped with. These maps are utilized so that a collision free trajectory is planned for each of the robotic manipulator's joints. Guiding the robot end-effector utilizing information provided by eyein-hand sensing as in the cases of [163] and [164,165] (parts of CROPS and SWEEPER projects respectively) is indeed a very common approach for visual servoing. In such approaches, information concerning the best approach direction to maximize target's detection rate may be highly valuable [166]. It is evident that visual servoing can be sensitive in camera calibration parameter uncertainties, as illustrated in [167] where the combined utilization of image-based visual servoing (IBVS) for the harvest target approach phase and position-based visual servoing (PBVS) for making fine adjustments in the robot's movement is proposed to cope with such uncertainties.
After the harvest target is reached and grasped by the robot's end-effector, its picking may be achieved via a detaching motion that the robot performs in order for the target fruit to be released from the tree's branch. This motion may vary with the obvious goal to detach the harvest target as easily as possible without causing any damage to it. Four basic picking patterns including horizontal pull, vertical tension, bending and twisting are performed in [168] in order to analyze the effects of different picking patterns on fruit detachment. The tension parameter, which includes the horizontal pull and vertical tension, is found to be the dominant factor during the detachment process, with the vertical pull possibly leading to the pull-out of the stem.
Optimized results indicate the horizontal pull with a bending and twisting motion as the potential optimum combination. Picking patterns for successful crop detachment are directly connected to the physical properties of harvest targets. To this end, works may be found in literature that focus their research to target crops themselves, investigating their rheological characteristics (deformation) [169] and physical properties regarding e.g. how much force may be exerted upon them without damaging or bruising them. In [169], apples are considered with the ultimate purpose of designing an impedance controller for the robotic system of [38] to compliantly pick them, while in [170] pumpkins are investigated in order to find an appropriate harvesting procedure that will not damage them. A major challenge in robot motion planning that one may easily consider is the possible harvest target motion (e.g. a fruit may move due to gusts of wind that may shake the tree's branches). In [171], such a problem is addressed by proposing a robust, imagebased, non-linear visual servo controller (along with its Lyapunov stability analysis) that regulates the end-effector to the target fruit's location, compensating for unknown disturbances like the fruit's motion.

HRI in Agriculture
Agricultural processes are mostly performed by humanoperated machines while few are handled by autonomous robots. However, there are applications that are difficult to be fully automated while the efforts in this direction appear to be on a long-term path. In this scope, HRI strategies in agriculture have attracted the scientific interest in order to address such complex problems, providing secure, faster and more productive solutions than the traditional approaches [172].
Several works in agricultural HRI literature focus on activities such as facilitating farming robots' navigation in fields and orchards, detection of fruits and vegetables, spraying [173]. In particular, different user interface modes for target recognition and spraying are explored in [174]. The robot navigation along vineyards can be handled by the user when needed, views from the on board camera sensors are visualized while teleoperation is achieved through three different interface configurations which use a mouse, a Wii remote 8 and a digital pen. Moreover, a web based UI for controlling autonomous vehicles for apple harvesting is proposed in [175]. The user can select the field area of interest, the vehicle's speed, and decide whether certain orchard's rows should be skipped or traversed multiple times. The benefits of the use of HRI strategies aided 8 https://www.nintendo.co.uk/Support/Wii/Usage/Wii-Remote/ Basic-Operations/Basic-Operations-243993.html -Last accessed: 22-11-2022 with visualization technologies for melon detection are analysed in [48]. Specifically, detection increased by 4% in comparison to manual detection and by 14% compared with a fully autonomous approach, reaching average detection rates between 94% and 100%. Moreover, the detection times were 20% shorter than the ones achieved manually.
Designing complex applications for HRI in agriculture requires deep understanding of the users' cognitive factors and the task demands. In this scope, using a knowledge engineering approach, the authors of [176] proposed a work model based on information obtained from operators to evaluate the mowing task in a citrus grove. Moreover, a method to estimate the operator's mental workload in multiple information presentation environment for agricultural vehicles is presented in [177]. The aim of the method is to facilitate the design and optimize the humanmachine interface resources to ensure highly-efficient and safe operations in the fields. Alternately, other works use sensors to estimate the human operator's cognitive status. Such example is provided in [178] where non-invasive sensors, such as EMG (electromyography), are used to study and analyse the tractor handling performance in agricultural fields while in [179] the tractor operator's driving characteristics are modeled through simulation so as to achieve better adaptation of the vehicle's dynamic properties to the driver.

Operation Planning for Crop Inspection and Harvesting
Production scheduling is a critical part of production management that has preoccupied the scientific community over the last decades leading to prolific literature. The researchers have investigated both static and dynamic scheduling scenarios [180] while the studied cases involve job-shops, multi-agent systems, robot-human cooperation, single robots operating multiple tasks [181] etc. Over the years, various approaches [182] have been used to deal with the optimization of task planning including particle swarm operations [183], artificial bee colonies [184], variable neighborhood search [185], Tabu search [186], ant colony optimization [187], evolutionary algorithms [188,189] and other heuristic approaches [190,191].
In the robotic agricultural domain, the operations planning applications include planning of harvesting actions, team coordination of autonomous vehicles, optimization of field area coverage and route planning. Moreover, humanrobot cooperation is also investigated, nevertheless, robots act, in these cases, as crop transporting vehicles scheduled to provide optimal support to human harvesters [192]. In particular, a stochastic simulation based on a finite state machine is used to evaluate human robot collaboration and provide to the strawberry-harvesting workers the optimal support with automated transport of the collected fruits. Regarding the optimal route/area covering planning, a general approach for fleets of autonomous vehicles is suggested in [193]. Taking into account different criteria, such as vehicles with different characteristics, field variabilities and the refilling possibility, the authors deal with this combinatorial optimization problem using the metaheuristic search of simulated annealing. Furthermore, in [194] multiple machines are scheduled to work on various operations in multiple areas based on the fields' readiness using Tabu and modified Tabu search technique. In [195], a mobile seed refilling system that feeds multiple planters during large-scale field operations optimizes its route using a stochastic solution based on a genetic algorithm. Moreover, the travelling salesman problem methodology is applied on [196,197] and [198] for the optimization of coverage planning for capacitated fields operations and route planning for sugarcane and avocado harvesting respectively. In [199], the field area coverage is handled by the users themselves through a UI that allows them to schedule the robots' routes taking also into account refilling stations to replenish their resources (e.g. pesticide and energy).
Apart from the applications focusing on the optimization of area/route coverage according to the fields' properties and the available means, several researchers explored the optimization of the actual harvesting actions. In this case, the robot is parked in front of the plant of interest and has to collect the available fruits with one or multiple arms. In [200], a multi-arm robotic melon harvester optimized its performance utilizing an extension of graph coloring. In particular, the robot scheduled the fruit collection and decreased the harvest time by selecting the most efficient actuators, number of manipulators and robot's velocity. Similarly, in [201], multiple arms of a kiwi robot collector are assigned with tasks based on a cluster approach. In [202], one manipulator is utilized to perform sensing and harvesting tasks simultaneously. The sequence of these actions is scheduled through the travelling salesman paradigm taking into account the actions' costs and the traveling time.

End-Effectors Development, Automated Gripper Design and Grasp Planning for Harvesting
In harvesting processes, the grasp quality is one of the most important factors for production quality [203]. In this scope, placed upon the harvesting robotic system's manipulator, end-effectors have been in the researchers' interest [204]. Being directly involved in the crop harvesting task, the design of these tools may require specific characteristics depending on the target crops type (e.g. shape, size). The required motion that an end-effector must perform in order to detach and harvest a fruit or a vegetable is usually considered within its design process, mostly attempting to mimic a human hand's motion when grasping and picking a fruit from a tree while other end-effectors include a cutting device for crop detaching, simplifying the harvesting procedure. Various sensors may be integrated in the endeffectors, giving feedback that helps the overall robotic system acknowledge whether the harvesting target is picked up or not.
In this scope, a two bionic finger end-effector equipped with fiber sensors detecting the best position for grasping kiwifruits as well as pressure sensors to avoid damaging the fruits during grasping is proposed in [205]. A multi-sensory configuration comprising distance, proximity, force and pressure sensors is considered in [206] for a universal endeffector for spherical fruits (e.g. apples, tomatoes, citrus). In particular, a two-fingered gripper singulates the fruit with a sunction pad while a laser device is responsible for cutting the fruit's stem. Inspired by the operating principle of wire-stripping pliers, the end-effector's upper and lower jaws cut and simultaneously hold the fruit by its peduncle in [207]. A novel six-fingered end-effector equipped with three infrared (IR) sensors for active cutting position control is proposed in [42] for autonomous strawberries harvesting. Strawberry harvesting is also considered in [208], where a robotic gripper with three fingers and pressure profile sensors is developed in addition to a hierarchical control scheme, based on a fuzzy controller for the gripper's force regulation and proper grasping criteria.
Besides the specifically pre-designed (non-configurable) end-effectors, gripper design automation is a new field of research that enables the existing equipment to be flexible and handle a larger range of tasks through reconfigurable grippers. A theoretical analysis taking into account the reconfigurability of the gripper is presented in [209], where a balance between customization and simplicity is suggested. Most of the studies in the literature do not facilitate completely autonomous finger generation, rather following one of the three main approaches: modular design, re-configurable design and customized design. In modular design approaches [210], there is a finger library where multiple types of finger geometries are stored. Then, based on a simplified geometry of the object to be grasped, the suitable pair of fingers is selected [211]. A different technique has been introduced in [212] where a grasping device adapts itself to the geometry of the object using a one DoF mechanism. These solutions while providing general and flexible solutions regardless the shape complexity, are computationally expensive because they iterate through each finger design for verification. In re-configurable designs [213], a gripper is designed with the possibility of re-configuring the fingers' positions. The approach in [214] employs a three-finger parallel jaw gripper where two of the fingers can change configuration while the third one is fixed. Such approaches while offering quick and simple solutions, require experts for the technical inputs of the algorithms. Finally, customized designs with custom fingers can be specifically generated for each object [210,[215][216][217].
Grasp planning is defined as a search process of the possible locations for grasping an object, aimed at identifying closured grasps. The research area for this matter is wide and unstructured, mainly due to the fact that none of the proposed approaches has been widely implemented and considered the reference for comparison. In [218] a clear overview of the methods and algorithms for grasp planning is presented and the authors divide the research done in this field into analytical approaches and data-driven (or knowledge-based) approaches.
Analytical approaches consist of methods that use geometric, kinematic and dynamic formulations in order to verify a grasp [219]. The most basic approaches for grasp planning consider the minimum number of required contact points for force-closure of objects with polyhedral geometry [220][221][222][223]. The downside of the proposed multicontact methods is the exhaustive search procedure. In heuristic methods, a large number of grasp positions are generated randomly [224] by defining a set of rules that have been tested on the object's model [225,226]. Then, unfeasible grasp candidates that do not fulfill the force-closure condition are filtered out. Although these approaches require low computational effort, they are usually limited to determining the local optima and don't take every possible grasp position into account. Another limitation of multi-contact grasp methods that have been presented is that they are computationally efficient for very simple objects. Furthermore, methods for general object shape, model the object as a cloud of 3D points [227][228][229] or triangular mesh [230] and can be employed even for online grasp planning. Such methods [229] are employed in [231] for a robot harvesting sweet peppers using cameras on the end-effector and a 9 DoF manipulator. Although such algorithms can be applied to many different object shapes with multiple facets and reduced computational effort, they still suffer from local optima problems like the heuristic methods.
In data-driven approaches the grasp candidates are ranked based on a specific metric after being sampled for a particular object. Such methodologies are generally based on existing grasp experience, generated by simulation or on a real robot [218]. A review of such approaches in [232] concludes that for each method the type and level of prior knowledge and the assumptions made about the objects being manipulated are the determining factor for their reliability. Grasping itself is highly dependent on the employed sensing and manipulation hardware, so the approaches cannot be compared with each other and benchmarked. In [233], automatic generation of possible grasp sets through a simulation-based method is described. This approach is different from most other studies as it focuses on the grasp process instead of the final grasp configuration. Furthermore, a date-driven method for industrial parallel-jaw grippers is proposed in [234]. The algorithm starts with generating a grasp database by sampling on nearly parallel surfaces, and a dynamics based grasp simulator is used for evaluating the grasps. Finally, the suitable grasps are measured by three defined quality metrics. Such approaches, although computationally cheap, require a great deal of manual input and are not applicable for automated or online grasp planning.

Related Research Projects
Several projects have been running during the last decade, regarding automated robotic agricultural applications, with Fig. 3 presenting an overview of them. Mostly funded by the European Union (Seventh Framework Programme, Horizon2020 9 ), they involve the development of robotic systems whose primary goal is to facilitate and/or execute various agricultural tasks, such as harvesting, spraying, monitoring/phenotyping. CROPS 10 ( Fig.3 -CROPS, subfigure taken from [73]) goal was to develop scientific know-how for a configurable, modular robotic system, including various modular tools (e.g. manipulators, sensors, sprayers, grippers) and adapting to several tasks. The robotic system developed within this project was capable of site-specific spraying and selective harvesting of sweet peppers and apples. Specifically, a 9 DoF manipulator was designed and tested for sweetpepper and apple harvesting, as well as close range spraying in vineyards [73]. Different end-effectors and a prototype canopy sprayer were manufactured for each different application. Sensory system included color cameras, a multispectral system for fruit detection and discrimination between different elements of plants and a Time-Of-Flight (TOF) camera providing fast acquisition of distances, thus enabling localization of harvest targets [235]. CROPS system was evaluated with regard to sweet pepper and apple harvesting in a commercial greenhouse and close range precision spraying in a vineyard [73]. Regarding sweet pepper harvesting, the developed system reported a 56-86% detection rate and a relatively low harvest rate of 33% with a high average harvest time of 94 seconds per fruit; however, the manipulator was intentionally moving with a low speed for safety reasons. Results for apple harvesting task, with special pruning firstly taking place on the crops, showed a 100% detection rate and a 72% harvest rate, requiring 15 to 30 seconds per apple. The system was successfully tested for close range target spraying in a greenhouse experiment with grapevines, reporting an 84% attained pesticide reduction rate. An early performance evaluation concerning specifically the sweet pepper harvesting task under field conditions was also presented in [236], reporting a difficulty in the successful picking of the harvest targets (6% rate) and a clear improvement (from 6% to 26%) after simplifying the crop, that is removing fruit clusters, leaves and other obstacles. Lastly, grape harvesting was attempted during the project as well, however with no success since grape grasping was not successful most of the time with the gripper prototype being unable to cut the thick stalks of grape bunches even at maximum pressure. SWEEPER 11 -Sweet Pepper Harvesting Robot ( Fig. 3 -SWEEPER, subfigure taken from http://www. sweeper-robot.eu/) was a H2020 project that built upon technologies, initially developed during the CROPS project. Its technical goal was to develop, test and validate a practical robotic harvesting solution for sweet peppers in real world condition. The developed robotic system involved a robotic vehicle with a single arm with a task-specific harvesting tool for the sweet pepper harvesting. For the identification of the fruits' location and the assessing of their maturity and quality, an RGB-D camera with custom made LED lightning fixtures was utilized, reporting simultaneously both color and depth information. The developed vision system included a Flash-No-Flash (FNF) controlled illumination acquisition protocol, facilitating target detection that is robust to ambient illumination effects [91]. The maturity and quality of the pepper was assessed based on color and shape features (perimeter/area ratio) measurements of the fruit. Before harvesting the fruits, a manual leaf-picking procedure was proposed, as in cases of image disturbances (e.g. leaves creating occlusions) the proposed system would not be able to identify the fruits. The idea of modifying target crops and their environment in order to make them "best fit" for robotic harvesting was considered during the overall project and methodologies to influence crop features were investigated and presented [58] such as cultivation practices, pruning, climate control and artificial illumination. The developed robotic system was evaluated [66] through the harvesting of 262 fruits along a 4 weeks period, where 104 fruits were part of a modified crop (low occlusion/cluster interference) and 159 of an unmodified one. The average achieved harvest time per fruit was 24 seconds, with logistics taking approximately half of that time. Harvest success rates were reported to be 61% for best fit (modified) crop conditions and 18% otherwise.
VINEyardROBOT 12 or VINEROBOT (Fig.3 -VINE-yardROBOT, subfigure taken from [237]) was an FP7 project, whose goal was the design and development of an agricultural robot for vineyard monitoring. Focusing on optimizing a vineyard's management, VINER-OBOT project's basis consisted of building a ground robot/vehicle, endowed with artificial intelligence and several non-invasive sensing technologies. These technologies included fluorescence-based sensors, RGB machine vision, thermography, Visible (Vis) and Near-Infrared (NIR) spectroscopy and were combined with canopy images and data acquisition processes, executed in real time via customized algorithms. Monitoring the vineyard's status and grapes composition e.g. amino acid concentration [238], the proposed robotic system provided key information that would help the farmers in crop yield estimation [239], vegetative growth and water status inspection [240].
Focusing on helping the winegrowers to accurately assess their grapevines yield, VINBOT 13 -VINyard roBOT project ( Fig. 3 -VINBOT, subfigure taken from [241]) aimed at developing an automatic robotic system that would monitor vineyards and provide useful relevant information and phyto-data in order to facilitate yield estimation and overall vineyard management. The developed robotic system involved an all-terrain autonomous mobile robotic platform with open source software. It was equipped with a set of sensors (e.g. color cameras, 2D laser rangefinders) capable of navigating through the field, while capturing and analyzing images and 3D data via cloudbased computing applications. The system's objectives included the estimation of the amount of leaves and grapes (leaf-to-fruit ratio), canopy features like height, volume, and exposed leaf area, other phyto-data throughout the entire vineyard as well as the generation of online yield maps, that would greatly help winegrowers optimize their management strategies. Promising an autonomous monitoring of 168 hectares three times a year, an ability of climbing slopes up to 45 o and an electrical power autonomy of 8 hours per day, VINBOT aimed at representing a powerful precision viticulture cloud-computing agricultural tool, helping winegrowers produce higher quality wines. The evaluation of the developed system [241,242] reported good estimation results with acceptable accuracy regarding canopy features, with the overall grape yield however being underestimated due to various existing occlusions (bunchon-bunch/leaf occlusions).
Part of ECHORD++ 14 FP7 project that funded various application-oriented research projects called "experiments", GRAPE 15 -Ground Robot for vineyArd monitoring and ProtEction (Fig. 3 -GRAPE, subfigure taken from [243]) aimed at creating a robotic system that would help with the vineyard monitoring and its overall protection. The project's partners included VITIROVER 16 , a French company involved with vegetation robotic management and manufacturing of herds of mower-robots for soil grassing management. GRAPE's main objective included the creation of enabling technologies that will allow precision agricultural practices, significantly reducing the negative environment impact of traditional farming (e.g. mechanical instead of chemical thinning, biological control instead of chemical pesticides). The developed robotic platform -UGV would detect plants, manipulate small objects and help with the vineyard's health monitoring. A considered application of the system involved the automatic pheromone dispenser distribution for mating disruption (pest management) in vineyards [74]. Specifically, the robot would monitor plants and their health through utilization of vision systems (multispectral camera, 2D laser sensor) and 3D real-time reconstruction; subsequently a biocontrol mechanism consisting of pheromone dispensers would be applied wherever needed for plague control. Additionally, a significant amount of research was conducted regarding the autonomous navigation of the robotic platform [243]. With the vineyard environment presenting lots of challenges (e.g. weather conditions, soil and general terrain morphology, vegetation), GRAPE investigated utilization of various sensors and methodologies to facilitate the robot autonomous navigation through the field, including wheel encoders, GPS, LiDAR sensor, inertial measurement unit (IMU), Gmapping, Google's Cartographer and others.
Aiming at facilitating asparagus harvesting and lowering its overall costs, GARotics 17 ( Fig. 3 -GARotics, subfigure taken from [244]), also part of ECHORD++ FP7, developed a prototype robotic system for selective green asparagus harvesting. The proposed system was able to (a) drive along an asparagus dam in the field, (b) detect asparagus stalks, identifying the ones ready for harvesting (involving threedimensional-point cloud processing from RGBD image acquisition) and (c) perform harvesting without damaging the stalks, proposing a harvesting mechanism that utilizes a multi-tools solution for increased productivity. Initial field tests showed the harvester's applicability, reporting an average harvester's velocity of 0.2 m/s with a mechanical harvesting cycle of approximately 2 seconds, an average of 5 harvested asparagus plants per meter with a two harvesting tools setup and about 90% success rate in the harvesting process [244].
MARS 18 -Mobile Agricultural Robot Swarms project, an ECHORD++ FP7 "experiment" as well, has developed a robot system for high-precision planting, which has also been tested in the field. It uses small robots operating in swarms (6-12 units covering approximately 1 ha/h) and a cloud-based solution to plan, monitor and accurately document precise planting of corn. Satellite navigation and data management in the cloud allows operations to be conducted round the clock, with permanent access to all data. The position and planting time of each seed is accurately recorded, while planning for the required field, for seeds, seed patterns and density is carried out via an app. The intelligent OptiVisor algorithm plans the robot's deployment based on the entered parameters, calculating the optimal paths for the units involved and the time required for completion of the job. Software updates for the system can be loaded "over the air", just as a remote diagnostic can be run conveniently and in any location via the smart device. The latest version of the MARS robots was unveiled to the public for the first time at Agritechnica 2017, with the entire system, including small robots operating in swarms and a cloud-based system control now being operated under the product name "XAVER" 19 ( Fig. 3 -MARS(XAVER), subfigure taken from https://www.fendt.com/int/xaver).
VineScout 20 (Fig. 3 -VineScout, subfigure taken from https://cordis.europa.eu/project/id/737669) was the continuation of the completed project VineRobot. The project's main goal was to industrialize, demonstrate, and take to market a field monitoring system (decision support system), embedded in a small-size and cost-efficient robot, regarding vineyards. Project's objectives included optimization of the robotic system's external design and internal electronics as well as the development of top performance navigation software and systems for protection and user friendliness. Conducted research within the project involved on-the-go utilization of Visible-Short Wave Near-Infrared spectroscopy [245] and hyperspectral imaging [246] as monitoring tools for grape composition (e.g. soluble solids, anthocyanins, polyphenols), thermal imaging for the assessment of vineyard's water status [247] as well as an investigation over the growing data-driven agriculture and the status of current advanced farm management systems [14].
Project ROMI 21 -RObotics for MIcrofarms (Fig. 3 -ROMI, subfigures taken from https://cordis.europa. eu/project/id/773875 -Results) develops an open and lightweight robotics platform for microfarms, found both in rural, peri-urban and urban areas, growing a large variety of crops (up to 100 different varieties of vegetables per year) on small surfaces (0.01 to 5 ha). ROMI's goal is to assist these farms in weed reduction and crop monitoring, reducing manual labor and increasing the productivity. Thanks to ROMI's weeding land robot, farmers will save 25% of their time, also acquiring detailed information on sample plants. This robot is coupled with a drone that acquires more global information at crop level, thus producing an integrated, multi-scale picture of the crop development that will help the farmer monitor the crops and increase efficiently harvesting. For this, ROMI aims to adapt and extend state-of-the-art land-based and airborne monitoring tools to handle small fields with complex layouts and mixed crops. In addition to the ground and aerial robots, ROMI also introduces a 3D scanner for phenotyping in indoor and outdoor environments, combining an RGB camera with a powerful image processing pipeline for 3D plant representation and analysis.
With grape harvesting remaining an open and challenging task, new contributions can be significantly helpful, further advancing the use of automated agricultural technologies in this particular high-value field, i.e. vineyards. To this end, ongoing EU funded project BACCHUS 22 (Fig. 3 -BACCHUS, subfigures taken from https://bacchus-project. eu/) is developing an intelligent, mobile robotic system involving two independent and cooperative robots: (a) the inspection robot will navigate through a vineyard, inspecting and mapping crops, collecting various useful data, e.g. regarding their health and maturity, via an embedded sensorial system; (b) the harvesting platform will perform human-like harvesting with the needed finesse, introducing a bi-manual concept in contrast to most existing one-armed/uni-manual robotic solutions, thus offering additional manipulability, facilitating further the overall procedure. The envisioned solution aims at precision farming, applicable not only in vineyards but in other crops as well, incorporating smart use of robotics for advanced control, navigation/motion, adaptation and learning, towards increased farm productivity and reduced manual labor.
An overview of the above projects and their target functionalities is summarized in Table 2, with some additional, technical information regarding their hardware/software characteristics as well.

Related Commercial Products
In addition to research works and projects regarding automated harvesting and agriculture, several commercial products have been developed by various companies in the last decades, illustrated in Fig. 4 (all subfigures taken from products' websites that can be found in their respective citations/references).
Harvest CROO Robotics [249] was established in 2013, introducing an automated harvester for strawberries. Other strawberry robotic harvesters have been developed by Dogtooth [250] in the UK, Agrobot E-Series [251] in Spain and OCTINION [252] with their "Rubion" robot. In addition to their harvest robot, OCTINION has developed other robots for various agricultural activities besides harvesting (e.g. "Curion" for crop scouting, "Lumion" for fighting powdery mildew on strawberries, "Fluxion" for the logistic handling of storage crates). An autonomous strawberry harvester mounted on their robotic platform "Thorvald" is tested by SAGA Robotics [253] as well. The Thorvald robot, in particular Thorvald I [248] and the further developed Thorvald II [254], is an innovative module-based robot that allows for different robots to be build utilizing the same basic modules, hence addressing a variety of agricultural applications (e.g. crate transportation, UV-treatment, soil sampling, weeding).
The EU funded MetoMotion [255] developed the Greenhouse Robotic Worker -"GRoW" robot for selective robotic harvesting of greenhouse tomatoes, identifying and locating ripe fruits and performing robust damage free harvesting. Initially designed for greenhouse tomatoes as well, "Virgo" is a robotic universal harvester from Root-AI [256], now acquired by AppHarvest, that can be configured to identify and harvest multiple crops of varying sizes including tomatoes, peppers, cucumbers and strawberries; AppHarvest [257] focuses on controlled environment agriculture, involving smarter in-door farming as well as precise and eco-friendly growing (e.g. recycled rainwater, chemical pesticide-free fruits and vegetables). ENERGID [258] has developed a robotic harvesting system for citrus reporting a 2 to 3 seconds harvest time per orange and a picking rate of 50%, while Abundant Robotics was in the process of launching their commercial robotic apple harvester as well; however, it was unable to develop the market traction necessary to support its business during the COVID 19 pandemic, resulting in its shut down 23 .
An interest has also been shown for grapevines, with VISION ROBOTICS Corporation's intelligent autonomous grapevine pruner [259] and naio Technologies "TED" [260]. Utilizing GPS-RTK navigation and with an autonomy of

Conclusions
A great amount of papers, projects and products may be found regarding the on-going research field of automated agriculture, focusing on autonomous harvesting but also on other tasks such as weeding, health monitoring, pruning, yield estimation and phenotyping. Apples, strawberries, tomatoes and sweet peppers occupy the majority of research interest, with grapes being targeted as well with respect

Conflict of Interests
The authors declare that they have no conflict of interest.
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons. . His main research interests include human-robot interaction, robot vision, service robot perception and cognition, affective computing, human motion, activity and behaviour analysis and modelling, safe and social-aware robot navigation, biosignals processing and sensor management, multimodal interfaces, machine learning and pattern recognition.
Dimitrios Tzovaras is a Senior Researcher (Researcher A') and the Centre for Research and Technology Hellas (CERTH) President of the Board. He received his diploma in Electrical and Computer Engineering (1992) and his Ph.D (1997) from the Aristotle University of Thessaloniki (AUTH) while he has been working as a Researcher since September 1999. His main research interests include visual analytics, 3D object recognition, search and retrieval, behavioral biometrics, assistive technologies, information and knowledge management, multimodal interfaces, computer graphics and virtual reality.
Dr. Dimitrios Kateris is an Assistant Researcher (Grade C') at the Institute for Bio-Economy and Agri-Technology (iBO) of the Centre for Research and Technology Hellas (CERTH). In 2015 he received his PhD in Agricultural Engineering from Aristotle University of Thessaloniki for his dissertation on "Intelligent system for fault diagnosis in agricultural tractor mechanical subsystems". Also, he holds a MSc in Agricultural Engineering and Water Resources from the same Institute (2009) and two BSc in Hydraulics, Soil Science and Agricultural Engineering from Aristotle University of Thessaloniki (2006) and from Technological Educational Institute of Thessaly (1999) respectively. His research interests lie mainly in the areas of Agricultural Robotics, Digital Agriculture, Agricultural Engineering, Decision Support Systems (DSS), Operations Management, Information Systems, Automation Systems in Agriculture, Machine Learning. His involvement with the above research areas has led to the co-authoring of over 47 papers in refereed journals, 2 books, 6 book chapters, 1 patent and more than 100 papers in international and national conferences. He has served as a regular reviewer in many international journals and conferences. Also, he has participated (as researcher, WP leader, partner, and coordinator) in more than 20 European and National Projects.
Prof. Dionysis D. Bochtis (M) is the Director of CERTH/iBO. Previous positions include: Chair Professor (Agri-Robotics) at the Dept. of Computer Science, University of Lincoln, UK, and Senior Scientist (Operations Management) at the Dept. of Engineering at Aarhus University, Denmark. He holds a PhD in Fleet management in bio-production systems, a MSc in Automation Control, and a B.Sc. in Exact Sciences (Physics). His primary research is Systems Engineering focused on bio-production and related provision systems including conventional systems with enhanced ICT and automation technologies and fully robotized systems. He has participated (as researcher, WP leader, partner, and coordinator) in 50 (32 EU) funded research projects. He has given 27 key-note speeches around on Systems Engineering topics. He is vice-chairman of CIGR (International Commission of Agricultural and Biosystems Engineers) Section V (Systems management) and he was the president of CIOSTA, 2011-2013 (Commission Internationale de l'Organisation Scientifique du Travail en Agriculture, founded in Paris, 1950).