Yield sensing technologies for perennial and annual horticultural crops: a review

Abstract

Yield maps provide a detailed account of crop production and potential revenue of a farm. This level of details enables a range of possibilities from improving input management, conducting on-farm experimentation, or generating profitability map, thus creating value for farmers. While this technology is widely available for field crops such as maize, soybean and grain, few yield sensing systems exist for horticultural crops such as berries, field vegetable or orchards. Nevertheless, a wide range of techniques and technologies have been investigated as potential means of sensing crop yield for horticultural crops. This paper reviews yield monitoring approaches that can be divided into proximal, either direct or indirect, and remote measurement principles. It reviews remote sensing as a way to estimate and forecast yield prior to harvest. For each approach, basic principles are explained as well as examples of application in horticultural crops and success rate. The different approaches provide whether a deterministic (direct measurement of weight for instance) or an empirical (capacitance measurements correlated to weight for instance) result, which may impact transferability. The discussion also covers the level of precision required for different tasks and the trend and future perspectives. This review demonstrated the need for more commercial solutions to map yield of horticultural crops. It also showed that several approaches have demonstrated high success rate and that combining technologies may be the best way to provide enough accuracy and robustness for future commercial systems.

Introduction

Farmers pay close attention to their farm yield because it directly impacts revenue. The more information they have on yield for each crop and each field on their farm, the better they can assess the impacts of decisions made during the crop growing season and improve management decisions for subsequent seasons. The interest in monitoring yield in arable crops triggered the development of conventional yield monitoring systems in the early 1990’s. The georeferencing of these data to enable yield mapping led to yield sensing being one of the most widely adopted precision agriculture technologies. Yield sensors are now commonly standard equipment on combine harvesters in many countries (Lowenberg-DeBoer & Erickson, 2019). Yield monitoring and mapping continues to be mainly used in combinable cropping systems. Yield monitoring systems for perennial and annual horticultural crops are still in the stage of infancy despite recuring spatial variability in crop yield (Fig. 1). Yet, yield monitoring systems can provide multiple advantages for these cropping systems as well. The following section explains multiple advantages of creating yield maps in perennial and annual horticultural crops.

Fig. 1

(Adapted from Manfrini et al., 2020)

Map of apple orchard yield as measured in kg of fruit harvested per tree. White dots show sampled trees. Map was created using kriging.

Why use yield sensing?

Yield maps represent the final output of all agronomic, meteorological, and management conditions that prevailed during a growing season. Their value is further realised when yield maps are used as part of the crop or field management decision process. Potential uses of yield map data include:

Harvesting logistics

Yield monitoring systems can supply real time information on harvested quantities enabling growers and others in the supply chain to make more informed decisions on storage, processing, and delivery of the crop. This is particularly important for horticultural crops which need to be processed soon after harvest and may reduce the cost of product handling, storage, yield loss, and labor cost (Ampatzidis et al., 2016; Bazzi et al., 2021). For example, for fresh produce crops that are processed soon after harvest, accurate information on yield volumes can allow factories and processors to prepare and open lines ready to process larger volumes of crop or close them down when lower volumes are expected.

Targeting low and high yielding areas

Yield maps (single or multiple years) can be used to identify the highest and lowest yielding areas of a field for further investigation (Blackmore, 2000; Khosla & Flynn, 2008). This information can be used to target soil or crop sampling (perennial crops) to understand yield-limiting factors. If the yield-limiting factors can be identified and eliminated or managed, then low yielding areas can be improved, resulting in ‘quick wins’ for all crops grown in the rotation. For example, localised areas of low pH that can be corrected by variable rate liming and areas of poor soil drainage that can be addressed by installing or repairing field drains (Oliver & Robertson, 2013). However, if yield-limiting factors are unmanageable, then production potential and associated input levels can be adjusted to achieve higher input use efficiency. An emblematic example of this strategy is to implement selective harvesting in viticulture (Bramley et al., 2011) when its is known that yield varies with the quality level of the grapes (Arnó et al., 2012).

Evaluating whether variable rate management is likely to be of benefit

Spatial yield variation can also be used to assess the benefits in exploring the causes of variation and implementing variable rate management (Lund et al., 2000; Muhammed et al., 2017; Pringle et al., 2003). Variable rate management is likely to be of greatest benefit in fields that are inherently variable (Koch et al., 2004). Where yield variation is relatively low, the cost of detailed sampling and variable rate management is less likely to be justified. Using yield maps, growers who are interested in adopting variable rate management can identify which fields are more likely to respond profitably (Hornung et al., 2006). For instance, yield maps can be used to calculate an Opportunity Index and to rank fields according to their likely success if for variable rate management is adopted (Tisseyre & McBratney, 2008), while vigour mapping and targeted sampling can form the basis of differential and selective harvesting approaches (Briot et al., 2015).

Creating management zones for variable rate strategies

Yield maps can be used on their own or in combination with other spatial data (e.g., soil maps or canopy vigour maps) to define field management zones, which can in turn be used as the basis for site-specific management (Hornung et al., 2006). Multiple years of yield data can be combined to help identify spatial yield patterns that are relatively constant over time (Blackmore et al., 2003; Leroux et al., 2018). However, it is not advisable to combine yield patterns from multiple crops in the rotation unless the crops have similar management and growth characteristics (Boubou, 2018).

Defining a yield potential map

A time-series of yield maps can be used to identify areas of constant or variable yield response and define maps of yield potential, which in turn can be used for site-specific management (e.g. Blasch et al., 2020). This approach is best suited in cropping systems where the input recommendation is dependent on the expected/target yield.

Creating site-specific gross margin and profit maps

Yield maps, together with commodity price information, can form the basis of site-specific gross-margin or net-margin maps when paired with site-specific management and the economic costs of this management (Koch et al., 2004; Massey et al., 2008). With uniform management, profit follows yield patterns. However, under variable management conditions, gross margin maps may deviate from yield maps. This deviation may become even more prominent if additional quality information (e.g., onion size class or grape Brix value) is included and yield segregation based on quality is achievable.

Calculating nutrient removal

Yield maps can be used to calculate spatial variation in crop nutrients (nitrogen, phosphorus, and potassium) removal. If the harvested crop phosphate and potash content is known, either from sampling and analysis or from use of standard figures, this can be combined with yield data to provide a map of crop’s nutrient uptake, which can be used as the basis for variable rate nitrogen, phosphate and potash fertiliser application (Inman et al., 2005; Sagoo et al., 2017).

On farm experiments

Yield sensing is a useful tool to evaluate on-farm experiments (Griffin, 2010; Marchant et al., 2019). Farmers can evaluate the impacts of farm experiments by using a yield sensing system that will provide yield data across all treatments. A detailed yield map is preferable over one aggregated yield value encompassing different treatments or having to add complexity by harvesting experimental plots one by one to measure yield. This approach is often used in arable research projects and could be applied to horticultural crops as well (Kindred et al., 2017; Whelan et al., 2012).

Traceability

Yield data is important for traceability and to increase safety of the agri-food supply chain (Costa et al., 2013). With globalization, food poisoning and contamination has become a global concern for human health (AL-Mamun et al., 2018). Traceability is recognized as a way to minimize impacts of food poisoning events and traceability down to the block (e.g., 3 m x 3 m) can result in lower amounts of product being recalled (Lupo, 2019; Shevchuk, 2019). Certain yield monitoring approaches can trace back harvested products by linking geo-coordinates to harvesting containers (e.g., a lettuce box).

Implementing digital agronomy in horticultural crops

Yield being the output result of farm management decisions, it is the necessary feedback for assessing the impact of the decisions made and how to improve crop management. Artificial intelligence offers the potential to analyse large volumes of data including environmental and management data (e.g., soil properties or N fertilizer rate) along with resulting output data (e.g., yield or moisture content) to help farmers optimize their decisions on the farm (Li & Yost, 2000). Yield sensing technologies in horticulture will enable this agricultural sector to fully enter the era of digital agriculture to improve environmental and economical sustainability of food production (Basso & Antle, 2020).

The specific context of yield sensing in horticultural crops

Both the commercial development and adoption of yield sensors for horticultural crops has lagged behind the adoption in arable cropping systems. There are several contributing factors to this slower commercial development and lower adoption rate.

Harvesting methods

One aspect of lagged adoption of yield monitoring systems in horticulture is related to the diversity in harvesting methods. Horticultural crops may be harvested mechanically, semi-mechanically or manually (Erkan & Dogan, 2019). Yield sensing will require different tools and strategies depending on the harvesting methodology and the crop type. Hand harvested crops are low technology and low throughput, potentially requiring complex systems using tracking devices coupled with counters or load cells to generate yield maps. Semi-mechanical harvesting systems are meant to benefit from the advantages of both hand and mechanical harvesting ensuring careful and thoughtful fruit and vegetable manipulations while transferring tedious and tiring components of the harvesting process to a machine. These systems offer better potential for the addition of yield sensing components on the human-assisting devices. Fully mechanized harvesters offer the greatest potential for high throughput yield monitoring but are often targeted only at processed fruits and vegetables. It is expected that, with the advent of robotics on farms and the increasing complications of migrant farm labor hire, mechanized harvesting will increase in fresh market fruit and vegetable harvest, create new opportunities for yield monitoring, and will also require adaptability of the systems with evolving cropping systems (Downing & Coe, 2018; Hennessy, 2019; Vougioukas, 2019). For instance, evolving farming machinery towards autonomous robots will require adaptation of yield monitoring systems.

Small field plots

Fruit and vegetable production areas are often smaller than arable fields (Lesiv et al., 2019; Yan & Roy, 2016; Zude-Sasse et al., 2016). For instance, large vegetable farms in Salinas Valley, California have an average field size of 4 ha despite managing up to 2000 ha of crop land (Cahn & Johnson, 2017). It has been demonstrated that spatial variability exists even in small fields (Cao et al., 2012; Kharel et al., 2019). However, mapping the yield of smaller fields may add complexities to data collection and usage. For instance, yield monitoring systems often generate more error near field boundaries associated with machinery turns (Beck et al., 1999). If there are more turns in proportion to the field size, this may result in a higher error rate for smaller fields (Luck et al., 2015). Therefore, if a yield mapping system is designed for small fields, it should account for this phenomenon and find solutions to lessen the impact of edges on data quality in order to maintain a good proportion of the map usable for decision making.

Plant architecture

Combinable crops often display similar architecture that is suitable for mechanized harvesting, i.e. upright architecture with the harvestable component clearly presented to the harvester (Baldanzi et al., 2003; Singh & Nimbkar, 2016). In contrast, fruit and vegetable crops often display a diversity of architectures that make the design of yield sensing systems limited in terms of the adaptability to other crops and other farms (Glancey & Kee, 2005). For instance, a yield sensing system designed to work efficiently for processed tomatoes is not suited to fresh market tomatoes, which are softer and need to remain intact. One possibility is to adapt the yield monitoring system to the crop architecture and the other possibility is to adapt the crop architecture to the yield monitoring system (Gongal et al., 2015; Robinson et al., 2013). Nevertheless, the vast diversity of plant architecture in fruit and vegetable crops encourages designs that are more adapted to the crop and harvest systems and thus less transferable to other crops.

Total yield versus marketable yield,

For combinable crops, typically all the harvested crop is sold, and the total yield approximates the marketable yield. This is also true for many horticultural field crops for processing because the yield and value of these crops are measured mostly in terms of the quantity rather than the quality. For some other field horticultural crops, e.g., root vegetables, and for fresh market fruits and vegetables, there may be a large difference between the total production yield and the actual marketable yield. To be marketable, the produce must meet some minimum quality thresholds, which will vary with crop type and target markets. Therefore, in many horticultural systems, quality is as important as quantity in determining crop value. In some cases, the total yield may be harvested from a field and subsequently segregated, typically associated with mechanical harvest, while in other cases only marketable fruit will be picked and non-marketable produce will be left in the field. Additionally, regardless of whether it is segregated in the field and/or in a packaging or processing plant, the marketable yield from a crop or field may be split between different markets, further complicating yield monitoring and the ultimate determination of site-specific crop value. Johnson et al., (2018) estimated the percentage of yield remaining in the field after harvest for eight crops and found that this percentage ranged from about 10–15% for cabbage and summer squash, as high as above 65% for cucumber, and even more than 85% for some watermelon. Yield left in the field is not recorded in governmental statistics but can represent high percentages for certain horticultural crops. Finally, in some cases production may be optimised but not-harvested due to external factors, such as a collapsed market or a lack of available labour to harvest the product at the proper maturity.

The distinction between marketable and total yield as well as crop value (e.g., onion size) should be accounted for in yield mapping systems in order to be valuable for decision making. These considerations are limited when working with broadacre commodity crops (e.g., moisture or protein content), but they can have a major impact for fruit and vegetable yield mapping, especially for fresh market.

Requires information on quantity and quality.

Quality parameters differ greatly between crops and even between specific markets for a given crop. In some case there may even be trade-offs between quality parameters (e.g., appearance vs. shelf-life) besides within-field trade-offs between the quantity and quality of production. Examples of quality parameters include visual indices, such as size, shape, color, and hull splitting, physical indices, such as firmness, juice yield and specific gravity, and chemical indices, such as acidity, total soluble solids (TSS) content, starch content and astringency (Erkan & Dogan, 2019). In addition, the presence of a deficiency, disease, or insect damage impacts quality (Cubero et al., 2015).

Different crops and different markets will require the sensing of different parameters. Furthermore, quality attributes may be measured with sampling or continuously, they may be measured in situ in the field or on a harvester or post-harvest in packing and processing facilities. They may be measured destructively or non-destructively. Some examples of technologies enabling measurement of quality parameters include the Delta Absorbance (DA) meter (Sintéleia χ, Bologna, Italy) using red and near-infrared optical signal to estimate chlorophyll content of apples and the Multiplex (Force-A, Orsay, France) using induced fluorescence to measure anthocyanins (Bramley et al., 2011; DeLong et al., 2016).

For most horticultural crops, the value and profitability of the crop will be determined by the marketable yield attained at a given price point. In this sense the trade-off in profit between producing a large amount of low-grade produce versus a small amount of high-grade produce is not always clear and is subject to many external factors. As compared to arable commodity crops, quality parameters are often a key component of harvest value and fruit and vegetable yield monitoring systems, which should measure key quality attributes in order to provide valuable information for economic and agronomic decision making (González-González et al., 2020).

Other consideration

Horticultural crops may present challenges for several other reasons as well. This section lists other aspects to consider along the ones above explaining the specific context of horticultural crops as compared to other field crops.

• Certain perennial horticultural crops that are biennial may demonstrate yield patterns and production that is influenced by the previous year’s production. For instance, a study using MODIS imagery in coffee production clearly detected the biennial fruit bearing effect on coffee yield (Bernardes et al., 2012). Incorporating this alternance in the analysis may be required to understand an area of low yield.

• Several horticultural crops are harvested by hand for lack of mechanized solutions that can handle the harvested product gingerly enough to maintain its integrity and marketability (Li et al., 2011). Therefore, a yield monitoring solution may appear economical and reliable to measure yield, but post-harvest produce condition, especially for fresh market produces, should be part of the considerations while development of the sensing technology.

• Several horticultural crops such a summer squash, peppers, and tomatoes for instance will be harvested several times based on fruit maturity during the crop growing season. Some tropical crops such as bananas are asynchronous, and as a result, plots are harvested throughout the year as the plants reach maturity. Similarly, in orchards, since fruit maturation is often staggered because of intra-plant/orchard variability (fruit is often collected in 2 or 3 picks), data are often collected manually and done on a per-plant basis (Manfrini et al., 2020a). This would involve that a yield mapping solution for those crops would have to offer a capability to cumulate several yield maps, or at least to cumulate measurements over a significant period, from the same field to provide a reliable map of the total yield production (Lamour et al., 2020). This may add complexity to the development of solutions for these specific crops.

• The tree habits and three dimensionality of the crops, the complexity of a bimembral plant (made of a scion and rootstock) and the perennial nature of tree crops require the adoption of sampling/monitoring techniques different from those adopted in other areas of intensive cultivation (Manfrini et al., 2009; Zude-Sasse et al., 2016).

• Orchards often require the presence of anti hail nets, and the diagnostics of physiological parameters connected to yield and plant productivity is often impeded during the season. Thus, crops protected by anti-hail nets may require specific techniques/protocols for yield data collection (Manfrini et al., 2019).

Given the diversity of products and cultivation methods, there are many other specificities to take into account when talking about horticulture. These features can be seen as opportunities for measuring or estimating yield. For instance, the fact that crops are weeded and highly organised in space (rows, vertical positioning with trellising, etc.) can make it possible to individualise each plant or tree by remote sensing. These particularities can also lead to strong constraints when, for example, the inter-row is occupied by another crop (grass) leading to difficulties in unmixing crop information by remote sensing or when dense crop canopies hinder global navigation satellite system (GNSS) satellite reception. All these opportunities or constraints explain the diversity of proposed solutions for horticultural yield sensing.

Overall, there are multiple advantages of generating yield map in perennial and annual horticultural crops and the technologies should be adapted to the specific context of these cropping systems. Multiple principles, techniques, and technologies exist and have been tested either experimentally or commercially to monitor yield in perennial and annual horticultural crop. The objective of this review paper is to provide an overview of yield sensing techniques and technologies for perennial and annual horticultural crop, including some examples of commercially available solutions.

Review of yield monitoring approaches in horticultural crops

Yield monitoring approaches can be divided into two main categories based upon whether they use proximal or remote sensing approaches for measurement. In addition, the proximal sensing approach can be subdivided into direct or indirect measurement principles. This section is thus divided into proximal (i.e., direct and indirect measurement) and remote sensing approaches.

Proximal sensing principles

The measurement principles listed below are either in contact or in proximity of the harvested produce. These systems are often mounted on the harvester or incorporated in the harvesting operations.

Direct measurement principles

Direct measurement principles return a direct estimate in the units in which the crop is marketed, most often mass, volume or unit count.

Weight measurement based on load cells

The direct assessment of the mass of crop being harvested can be done by using load cells. Load cells are electro-mechanical sensors that convert the force (mechanical energy) applied to the sensor into an electrical signal. The electrical signal can be calibrated and is linear with respect to the mass required to generate the force.

In horticultural crops, there have been 4 main ways that load cells have been deployed to generate yield data:

• 1. As a weighbridge within the harvester to measure the instantaneous continuous mass of the harvested crop as it is moved around the harvester. The most common place for this type of sensor is on a discharge conveyor belt.

• 2. Behind an impact plate on the harvester, typically again at a point of transfer between compartments in the harvester or to a trailing, support container. This also provides an instantaneous measurement of mass.

• 3. Beneath on-board storage bins or trailing bins to measure the real-time cumulative mass of crop as it is being harvested.

• 4. On equipment that collects harvest bins or boxes in the field to provide a discrete or cumulative mass for a portion of the field.

The first approach has been the most commercially adopted with current systems available for root crops (RiteYield system, Greentronics, Elmira, ON, Canada) and for grapes (Grape Yield Monitor, Advanced Technology Viticulture, Joslin, SA, Australia). It has also been demonstrated successfully in other crops including processing tomatoes (Pelletier & Upadhyaya, 1999) and sugarcane (Cerri & Magalhães, 2005), but are not yet commercialized. The commercially available yield monitors are designed to be retrofitted and should be applicable to a wide range of vegetable and fruit crops that are mechanically harvested and use conveyor belts.

A clear advantage to these systems is an ability to directly weigh the total mass of the crop. In some crops, particularly in well matured grapes, the off-loaded crop may contain a significant proportion of juice (crushed berries) as well as whole berries and clusters. Load cells do not distinguish between the forces exerted by liquid or solid crop components. Similarly, they operate irrespective of the density of the crop, which may vary greatly between varieties or even with harvest date in many crops, including potatoes. While this is advantageous for a yield (mass) measurement, it does have a limitation if an assessment of quality is needed.

The placement of a weighbridge within a moving, vibrating machine in often uneven terrain is not without its challenges. Separation of the yield signal from the background noise is critical for accurate yield measurement (Boschetti et al., 2013). Initially this issue caused difficulties in developing systems where the signal to noise ratio was similar, e.g. grape crops. As a result, the first load cell yield monitors were targeted into field horticultural crops such as for processing tomatoes and potatoes (Davenport et al., 2002; Demmel & Auernhammer, 1999; Pelletier & Upadhyaya, 1999) with very high yields and a strong difference between the yield and noise signals. Signal processing advances have meant that issues with a low signal to noise ratio can be solved in the electronics and load cell sensors are now effective at even very low off-load flow rates (Taylor et al., 2016). Errors associated with background vibrations and rolling/pitching can be mitigated by regular calibration and the use of a null load cell system to measure non-crop forces on the harvester. Changes in the speed and the tension of the conveyor system will affect the yield monitoring system and are common in harvesters. Bias and drift over time is therefore an issue and daily calibration (against an empty conveyor) is recommended. Despite this, when installed properly, regularly calibrated and operated correctly (both the harvester and the yield monitor), accurate high quality yield data can be obtained (Arslan & Colvin, 2002; Davenport et al., 2002).

The other main issue with weighbridge-type systems is that they measure everything that passes over the load cells, including foreign material and non-desirable crop material. This is especially problematic for root vegetable crops harvested under wet conditions. The amount of soil attached to the crop or the number of stones and soil clods that pass through the harvester will affect the mass measured. These effects have been shown to render the yield data unreliable, with 10% errors reported even when operated correctly to manufacturer specifications (Whelan & Mulcahy, 2017), and wet conditions often generating no usable yield data due to the large amount of soil adhering to the tubers during harvest (Davenport et al., 2002; Taylor et al., 2019). With dry, sandy soils at harvest, this is less of a concern.

The impact plate sensor is the most common sensing system in cereal crops; however, it has not gained wide acceptance in horticultural crops. This is perhaps because of a perceived reluctance to cause any additional damage to the fruit/vegetables by deliberately hitting the crop against an impact plate. Despite this, impact-plate sensors have been shown to be accurate for onion (Qarallah et al., 2008), citrus (Maja & Ehsani, 2010), processing tomatoes (Upadhyaya et al., 2006) and potatoes (Kabir et al., 2018), although drift was still identified as a major issue. The yield monitoring system of Maja & Ehsani (2010) in citrus, using an impact plate, enabled a prediction accuracy of 92.2% between computed and actual fruit weight under field conditions.

The use of load cells under harvest bins to measure the cumulative yield has been tried in several crops (peanuts - Vellidis et al., 2003; grapes - Taylor 2004; pecans - Rains et al., 2002; strawberry - Anjom et al., 2018; tomato - Abidine et al., 2003 and sugarcane - Cerri & Magalhães 2005) but was not commercialised. The system in Vellidis et al., (2003) was monitoring peanuts weight falling in a harvesting cart in real time. This yield sensing method provided a prediction accuracy between 97 and 98% under field conditions based on a trailer load unit. Pea viners in the United Kingdom used load cells in the collection tanks of the viners (Fig. 2; AHDB, 2018). In general, load cell-based systems can provide data with a low error over large scales; however, instantaneous yield measurements are difficult to extract thus limiting their usefulness for fine-scale management (Porter et al., 2017). In large crops (e.g., pumpkin or watermelon), the systems need to be able to operate under conditions of very light (empty) and very heavy (full) bins that exhibit different vibration and background noises as the bins fill. This requires specialised load cells and complicated signal processing (compared to the weighbridge and impact sensor approaches that have a much narrower range of target mass values) and for these reasons cumulative yield systems have been largely abandoned. There has been some recent renewed interest in these sensors in high value strawberry crops. Anjom et al., (2018) designed an affordable strawberry yield sensing system using load cells mounted to a light-weight strawberry-picking cart and connected to a Real Time Kinematic Global Navigation Satellite System (RTK-GNSS) receiver. This system mapped the yield of a strawberry picking cart as it went through the field and every time fruits are added to the cart, the additional weight is recorded along with the geo-coordinates. Even though this design was low tech (i.e., PVC pipes and Arduino controller), the prediction accuracy was 95.2% based on the full tray under field conditions.

Fig. 2

Yield map of a vining pea (Pisum sativum L.) field. Yield was measured with a retrofitted load cell in the collection tank of the vining pea harvester and yield data was linked to a GNSS feed to create a yield map

The final approach with load cells is to use them to batch weigh harvested crop. Unlike the previous approaches, a continuous, on-the-go, measurement is not recorded. Instead, bins or baskets within a field are weighed and geo-located. This is suitable, with effective track-and-trace technology, for hand harvested crops and a system for citrus groves was developed though not commercialised (Whitney et al., 2001). Similar systems are marketed on specific models of grape harvesting machine of New Holland (Braud, Coex, France) and Grégoire (Chateaubernard, France) in order to meet logistical and traceability requirements.These systems have the advantage of static weighing, providing more accurate discrete measurements; however, the yield data spatial resolution is considerably reduced. Yield maps showing clear trends in the data have been obtained from these types of systems (e.g., Praat et al., 2003; Schueller et al., 1999) but this approach has never attracted strong industry attention. It could suit large tree crops with individual, stand-alone trees (production units) that are harvested singularly.

High throughput volumetric measurement based

Yield of horticultural crops is often measured on a volumetric basis such as bushels pallets, boxes, sacs, etc. When the bulk density of horticultural produce, such as root crops or fruits is known, volumetric measurements can be used to estimate the mass. Variations in bulk density is often related to crop stresses, such as freezing, or to variation in fruit maturity and bulk density is assumed to be stable in commercial crops (Jadhav et al., 2014; Lorestani & Tabatabaeefar, 2006) observed a 6–10% coefficient of variation in two varieties of kiwifruit bulk density.

Volume is typically measured in harvesting systems by five main approaches (Moreda et al., 2009):

• 1. measurement of the volume of the gap between the fruit and the outer casing of embracing gauge equipment.

• 2. measurement of the distance between a radiation (e.g., sound or light) source and the object’s (e.g., fruit) contour.

• 3. measurement based on obstruction of light.

• 4. measurement based on active or passive two-dimensional (2-D) machine vision systems.

• 5. measurement based on active or passive three-dimensional (3-D) machine vision systems.

Volumetric sensing using ultrasonic sensors were an early form of commercialized horticultural yield sensors for use on discharge conveyor belts (Sartori et al., 2002; HM-500, HarvestMaster Inc., Logan, UT, USA). Issues with calibrating the sensor to varying produce density limited its effectiveness and these sensors have been superseded by load cell sensors (Taylor, 2004). More recent approaches for high-throughput fruit and vegetable sizing on a conveyor belt have focussed on the use of Light Detection and Ranging (LiDAR) systems and the use of 2-D vision (Jadhav et al., 2014, 2017) measured the volume of harvested fruits using a LiDAR to estimate total mass of harvest. The LiDAR sensed the conveyor belt from above with the LiDAR field-of-view perpendicularly to the fruit flow. The system showed a prediction accuracy of 93% under laboratory conditions, while the prediction accuracy was 90% under field conditions. Reduction in accuracy was attributed to belt speed variations, which can be accounted with a belt speed sensor system. This system thus shows promise for commercialisation.

Indirect measurement principles

Indirect measurement principles are all measurement systems that estimate a parameter that is related to the unit in which the crop is marketed as opposed to measuring the value directly.

Capacitance measurements based

One approach that has been investigated for high throughput yield monitoring is the use of capacitive sensors. Capacitance is the ability of a material to store electrical charge. The capacitance of an object (capacitor) can be measured using two conductive plates (electrodes) separated by a gap (Fig. 3). With fixed plate sizes, area and spacing, the capacitance recorded will be affected by the dielectric constant of any produce in the gap between the electrodes. In theory, the mass of the plant product with a relatively stable water content will be correlated with the capacitance. Research efforts have succeeded in using capacitance to measure the yield of different plant products, such as forage crops, sugar beet, hops, potatoes and chopped maize biomass (Kumhála et al., 2009, 2010). Under field harvest conditions, prediction accuracy of hops reached 96% (Kumhála et al., 2013).

Fig. 3

Schematic of a capacitive sensing system mounted on a conveyor belt transporting harvested produce

Machine vision

Machine vision-based systems consider the ability of cameras and other optical technologies to capture raw images and/or videos and extract useful and actionable information. In precision horticulture, there are different areas where machine vision is applied, which include crop growth and monitoring, yield mapping, and yield forecasting (Gongal et al., 2016; Li et al., 2016; Song et al., 2014). For the latter issue, different methods, approaches, technologies, and sensors have been investigated. Radiometric approaches have been largely discussed by Jiménez et al., (1999) and later by Kapach et al., (2012) and exploit the light reflectance differences between crop and foliage in the visible and/or infrared spectrum. Unfortunately, radiometric approaches have issues related to the plant/fruit growth habit and in particular, the visibility of the crop in the canopy and light conditions including changes of sunlight intensity and shadow effects. Illumination can be corrected by using artificial light sources to improve the accuracy of crop detection (Payne et al., 2014).

Machine vision was used by Roy et al., (2019) to develop a method based on a monocular camera for sensing yield in apple orchards and achieved a prediction accuracy of 92.0–94.8% with different datasets. Machine vision used in other tree crops has shown high prediction accuracy. For instance, prediction accuracy of fruit counting using machine vision was 92.3% for mangos and as high as 98.4% for citrus considering the fruit in the scene (Annamalai et al., 2004; Payne et al., 2013). Machine vision-based fruit counting usually underestimates actual fruit numbers because some fruits are hidden from the camera by tree branches and canopy (Annamalai et al., 2004; Payne et al., 2013).

Occluded and obstructed fruits among the canopy are more problematic and various approaches have been proposed. These include:

• i. Objects shape analysis for localizing spherical fruit with a smooth surface, has proven effective for detection of apples (Kelman & Linker, 2014), with 94% of fruits correctly detected in the scene and 14% false positives, but was not transferable to other fruit shapes.

• ii. Analyzing and transforming image regions into recognizable features spaces and training classifiers to associate them to either fruit or background objects, such as leaves or orchard/field structures (Bargoti & Underwood, 2017). As a smart technique to overcome this issue, forced air flow was also used to avoid occlusions (Gené-Mola et al., 2020a). This analysis is often performed through semantic image segmentation creating a pixel-wise classification over the image with subsequent processing to group adjacent pixels into individual whole-objects of interest. At the same time, the detection search area can be diminished using a lower-level image analysis to find regions of interest in the image, such as a hidden fruit within the canopy, followed by a high-level feature extraction and classification (Bargoti & Underwood, 2017; Gené-Mola et al., 2021).

The above studies generally analyzed images using “traditional” computer vision approaches. With continuous improvements in computational power, in particular with the use of processors optimized for matrix-similar data processing and big data calculations, many deep learning (DL) and convolutional neural network (CNN) algorithms and methodologies improved yield estimation (LeCun et al., 2015). An example is described by McCool et al., (2016), who developed a model for sweet pepper detection in field condition that adopted the faster region-based CNN (R-CNN) model. The model, after training, was able to achieve a higher detection accuracy (69.2%) in the detection of fruit and yields in real-world farms than the average detection accuracy of people (66.8%). DL models require large volume of data for training and validation for high accuracy (Krizhevsky et al., 2017) and it is well known how in CNN analyses a better result is achieved by more pictures of the object of interest. Rahnemoonfar & Sheppard (2017) developed a CNN architecture for counting tomatoes. To overcome the problem of a low amount of training data, they have generated a set of synthetic images to train on. The model was then tested on real images and showed an accuracy of about 80 − 85%. Similar experience was described in the study presented by Bresilla et al., (2019) on apple fruit with the use of deep CNN architecture based on single-stage detectors. In this study and in Koirala et al., (2019) it is also underlined that DL techniques eliminate the need for hard-code specific features for specific fruit shapes, color, or other attributes. A more recent study by Gené-Mola and colleagues (2020b) presented a new methodology for fruit detection and 3D location. The study developed a 2D fruit detection and segmentation using Mask R-CNN instance segmentation neural network then generating a 3D point cloud of detected apples using structure from motion photogrammetry. A projection of 2D image detections onto 3D space was used to remove false positives using a trained support vector machine.

Machine vision can also provide insights about harvest quality parameters. Chinchuluun et al., (2009) demonstrated that they were able to count the number of citrus and their size using machine vision while monitoring fruits on a conveyor of the harvester. They found an R² of 0.96 when comparing fruit weights to fruit diameter obtained by machine vision. This was an early demonstration that crops commercialized on a mass or volume basis could be monitored using machine vision. This demonstrates the potential of machine vision for sensing yield on the basis of both quantity and quality. Both of those aspects are important for crops, such as onion, that are priced based on their calibre (i.e., diameter). One commercial system offers the possibility to measure “color profile, size and quantity” of fruits for single trees in an orchard (Harvest Quality Vision™, Croptracker, Kingston, ON, Canada). Another similar example was described by Bazame et al., (2021). In this study, a computer vision model to detect and classify coffee fruits and map the fruits maturation stage during harvest was implemented. The fruit detection and classification were performed on framesets of images coming from a camera placed on the harvester using the object detection system YOLOv3-tiny. The model divided the fruits in three classes: green or unripe fruits; cherry or ripe fruits and raisin or overripe fruit. The average precision on the set of images was 86%, 85%, and 80%, respectively.

Deep learning or deep neural networks are now a well-established and successful tool in machine vision. One of the main advantages of deep learning compared to other image processing pipelines is that there is no need for feature engineering (Chollet & Allaire, 2018). Feature engineering is the process of defining parameters from image data as inputs for yield estimation models, for example. Deep learning has been demonstrated to be a promising approach to fruit counting and yield estimation (Keresztes et al., 2018; Sa et al., 2016).

Fusion of multiple sensing principles in one system

The combination of multiple sensing principles could potentially improve prediction accuracy (Moreda et al., 2009). Multi-sensor fusion can improve estimation accuracy by adding complementary sources of information such as vision systems with thermal or infra red camera, LiDAR data or ultra-sonic data Bargoti & Underwood, 2017; Bulanon et al., 2009; Hung et al., 2013; Underwood et al., 2016; Genè-Mola et al., 2020a; Wang et al., 2013). In their review of vision systems for yield sensing of tree crops, Ghatrehsamani & Ampatzidis (2019) suggested sensor fusion as the way forward to overcome sensing technologies weaknesses by compensating with other technologies. A clear example is given by the threshold algorithms applied to thermal imagery to count and report fruit morphological characteristics captured under natural environmental conditions in the orchard (Stajnko et al., 2004) or artificial conditions created by spraying water (Gan et al., 2018). This method exploits the higher thermal holding capacity of the fruit, due to its higher mass, compared to the leaves, but relies on all the fruit being exposed to equal or similar levels of solar radiation. Other multiple sensing systems exploit 3D orchard reconstruction using techniques as stereoscopic imagery, laser scanners or Time of Flight cameras (Jiang et al., 2008; Gené-Mola et al., 2020a; Si et al., 2015). These sets of sensors are consistent with varying lighting conditions thus more widely applicable to field conditions. The resulting 3D data structures can be analysed with various algorithms to exploit shape-related features of objects both in the image plane and 3D space and have been used to detect fruits (Nguyen et al., 2014; Tao & Zhou, 2017) and develop a color-agnostic fruit detection framework (Barnea et al., 2016). A recent study by Gené- Mola and colleagues (2019) applied the 3D data coupled with the intensity of the returns of a LiDAR sensor to identify the fruits in the scene due to the higher intensity of the returns impacting on fruits and the round shape created by those impacts on fruits due to the spheroid shape of fruits. By applying this technique, a localization success of 87.5%, an identification success of 82.4% were obtained compared to the total amount of fruits on the tree. These detection results are similar to those obtained by traditional machine vision systems, but with the advantages of providing direct 3D fruit location information, which is not affected by sunlight variations.

Radar sensors

Radio detection and ranging (Radar) sensors use high-frequency electromagnetic echo signals to detect properties about an object. These active sensors emit a waveform at a specific power level via a transmitter, and this waveform is then reflected on the target and captured by a receiver. Radar sensors can be used to characterize the surface (e.g., narrower wavebands reflecting on the surface) or the texture (i.e., wider wavebands penetrating the surface) of a target depending on the horizontal and vertical radar beam widths of the signal. This principle can be used to characterize the Earth’s surface (Synthetic-aperture radar) or to detect objects below the ground surface (ultra-wideband radar systems) among other possibilities.

Synthetic-aperture radar

Satellite (Ballester-Berman et al., 2012; Burini et al., 2005) or airborne (Burini et al., 2008; Schiavon et al., 2007) radar sensors have been already used in horticulture, mainly for vigor measurement and mapping of vineyards. More recently, Henry et al., (2019) reported the accurate estimation of grape yield from a ground-based 3-D radar as a contactless proximal sensor. The 3-D radar was built from the beam scanning of the canopy aiming at estimating the mass of grapes from the computation of appropriate statistical estimators derived from three sensors radar returns at three different frequencies. The proposed approach detects most grapes in the scene of interest, even if the grapes are partially or totally hidden by leaves, shoots or other grapes. Although very exploratory, this approach is promising to produce an estimate of grape yield and its spatial variability a few days before harvest. Indeed, it allows to overcome the limitations of current optical sensors that assume the visibility of all or part of the clusters (Abdelghafour et al., 2017).

Ultra-wideband radar systems

Ultra-wideband (UWB) radar systems have been studied for their potential to map root crop yields while they are still in the ground. Konstantinovic et al., (2008) used UWB radar to detect the presence of sugar beets in the soil. Combined with a GNSS receiver, they were able to map sugar beet production. The challenge of such a system lies in the differentiation of aggregates and soil products. The system was tested under field conditions, trying to detect the location and size of sugar beets in the soil and found above 80% correlation between the mass of sugar beets and the amount of backscattered energy. With a visual detectability of beetroot on radargram above 90%, the accuracy of this method may improve using machine vision, which has not been investigated in Konstantinovic et al., (2008).

Unit-based yield monitoring

The case of mechanized harvesting

Vehicle tracking refers to automatically tracking and determining vehicle location in real time using a GNSS receiver. The collected positioning data assists machinery operators, growers, and managers in several activities including, but not limited to, monitoring yield, developing yield maps, assessing machine productivity, and determining distances between points of interest. The typical mechanical harvesting operation consists of a harvester and a series of tractors pulling wagons/trailers travelling alongside the harvester. The harvester continuously or periodically unloads material into a wagon/trailer that is either driven to the processing/packaging plant or loaded onto a semi-trailer for delivery. Based on the movement of the machines and the harvesting operation process, logic can be developed to monitor these interactions and to make several assessments from the location of each machine, such as yield estimates and productivity assessments (Momin et al., 2019). Yield maps can be developed by correlating yield to wagon/trailer fill event distances so that the mass of the harvested crops of each wagon/trailer is divided by the fill event distance and row width to determine yield (M L^− 2). When both the harvester and machine traveled in parallel, and the harvester is within its conveyor arm radius of the machine for a prolonged duration, the harvester unload crops into the wagon/trailer. Based on this condition, the fill event distance can calculate for each specific machine. The logic and detail steps for developing the yield map using machine state data of a harvesting front is presented in Fig. 4. The main challenges for producing high accuracy yield maps using harvesting fleet tracking (i.e., harvester and weight wagon/trailer combo) include the requirement of sufficient density of data to characterize variability and calibrated weigh wagon/trailer.

Fig. 4

Adapted from Momin et al., 2019

Visualization of a fill event (i.e., the harvester and tractor travel next to each other in a parallel path while the harvester unloads sugarcane into the wagon/trailer) and the event radius surrounding the harvester, (A) and the resulting yield map (B).

The case of manual harvesting

Manually harvested crops are often collected in bags, bins, or boxes as a first step in harvesting before being collected and transported in a second step. These harvest units are usually filled to capacity and have a relatively stable mass when full. Counting and locating such units in a field or an orchard can thus inform on yield spatial distribution considering that the larger coverage area for a bag, the lower the yield (Colaço et al., 2020). Units can be georeferenced with a GNSS receiver during filling or when collected or could potentially be georeferenced using an unmanned aerial vehicle (UAV). Produce can also be boxed on-the-go during manual harvesting process, for example with broccoli or lettuce. In these systems where boxes are manually filled at the front of the harvester and placed on a conveyor to be stacked on a platform, boxes can be counted using a limit switch or an optical sensor for instance and each count georeferenced (Panneton et al., 1999). The time required to harvest can also be an indicator of yield. A study conducted in hand harvested coffee in Brazil showed that there was a linear relationship (R² = 0.86) between the time required to harvest a plant and the volume of fruit harvested (Oliveira Faria et al., 2020).

Citrus, mostly oranges, have received significant attention for yield sensing. In Florida, a system was implemented to weight palette bins collected from the field using a hydraulic lift with load cells to estimate the weight and to georeference them using a GNSS receiver. In such cases, the assumption is that each bin was filled from surrounding trees and the yield was projected by dividing the weight by the area covered by each bin (Schueller et al., 1999; Whitney et al., 1999). Alternatively, larger bags have been located in fixed points in the alleyways and weighed to generate yield information (Colaço & Molin, 2017; Molin et al., 2012). A similar system has been employed in olive orchards (Fountas et al., 2011). For orchards that are trellised, such as apple and pears, individual trees cannot be harvested and weighted as tree branches overlap. However, fixed distances along rows can be harvested, weighted, and again related to an area to generate a yield map (Aggelopoulou et al., 2009; Konopatzki et al., 2016; Liakos et al., 2017; Vatsanidou et al., 2017). A similar approach for yield monitoring has been conducted for measuring grapefruit (Peeters et al., 2015; Ünlü et al., 2014). Some studies have also measured individual tree yields for dates (Mazloumzadeh et al., 2010), pears (Perry et al., 2010), apples (Talebpour et al., 2019) and cacao (Carvalho et al., 2016). The latter case yields were summed over an early-harvest (March to August) and the main harvest period (September to February) per tree.

Georeferenced yield monitoring for manually harvested berries and manually-harvested vegetables during harvest is limited, probably as the measurement of yields from single plants or defined number of plants is not feasible. Typically, yield from fixed areas are used, for example Pozdnyakova et al., (2005) measured the yield in a cranberry field using a 0.3 m x 0.3 m frame while Akdemir et al., (2005) used a 10 m x 10 m frame for dry onions. Fountas et al., (2015) measured the yield of watermelons by weighing the crop from fixed blocks within a field.

It is possible to map the yield of handpicked horticultural crops with limited investment in technology, in many cases with just a handheld GNSS receiver.

Using radio frequency identification systems for yield tracking

Horticultural crops, and in particular perennial crops, have certain characteristics that may be conductive for the use of radio frequency identification (RFID), notably: (1) possibility of GNSS signal loss due to heavy canopy cover, (2) strong spatial organization of cultivated plants (rows, pots, etc.) and (3) long lasting and high value plants. The RFID system consists of a transponder tag and an RFID reader and works over a short range, typically < 10 m. Features in orchards, such as individual trees, panels or bays, can be georeferenced and given unique IDs. Similarly, harvesting units (bins or bags) can be tagged and then automatically associated with the harvested location (tree, panel, bay, etc.). Once weighed, the mass of crop in a harvest unit can be spatially transferred back to a location in the field and yield maps generated for a variety of crops including apples, kiwifruit, grapes and peach (Ampatzidis et al., 2008, 2009; Ampatzidis & Vougioukas, 2009; Praat et al., 2003). More recently, this approach has been enhanced by utilizing portable weighing units and RFIDs that permit the creation of yield maps, the monitoring of picker efficiency (associate pickers with buckets and fruit weight) (Ampatzidis et al., 2012, 2013) for sweet cherries and apples (Fig. 5). The same system can be used for the evaluation of harvest efficiency in orchards with various tree architectures (Ampatzidis & Whiting, 2013), and to streamline harvest operations and optimize field logistics (Ampatzidis et al., 2014). Furthermore, the RFID or barcode associated with a harvest bin is tracked into a packaging plant, then quality attributes as well as yield can be spatially mapped within fields (Praat et al., 2003; Taylor et al., 2007).

Fig. 5

Illustration of the real-time Labor Monitoring System (LMS) designed using a load cell and a RFID system. (CU = computational unit, RFID = radio frequency identification) (From Ampatzidis et al., 2012)

Remote sensing measurement principles

Remote sensing has been proposed as an indirect source of information to estimate field yield and its variability in horticulture, such as for potatoes (Li et al., 2020), tomatoes (Johansen et al., 2019), mangoes (Sarron et al., 2018), peaches (Horton et al., 2017), apples (Xiao et al., 2014), melons (Zhao et al., 2017), eggplants and lime (Moeckel et al., 2018), or vines (Di Gennaro et al., 2019b). It should be noted that most research reports refer to the use of UAV multispectral images with resolution lower than 0.1 m (Fig. 6). Few papers refer to airborne multispectral images with a spatial resolution of approximately 0.5 m (Carrillo et al., 2016; Hunt et al., 2018; Miranda et al., 2018), while fewer papers refer to the use of satellite images, among which Rahman et al., (2018) propose the use of satellite images with a resolution similar to that of UAV or airborne images (approx. 0.3 m) while Sun et al., (2017), investigated satellite temporal series at a resolution of 30 m for large vine grape fields. One of the specificities of perennial horticultural crops is that plants or rows of plants can be individualised. These are generally weeded crops. As a result, most approaches are based on individualising the plants (or plants within the rows) in order to extract vegetative indices or features corresponding to individual plants. This specificity justifies that most research uses very high-resolution images to be able to segment the plants on the images.

In horticulture, most common approaches hypothesize that plant yield is correlated with photosynthetically active biomass at a key phenological stage (KS) of the crop. As a result, an estimate of the biomass at KS from a vegetation index (VI) derived from a multispectral image is assumed to be a good estimate of the yield and its within field variability. The relationship resulting from this approach is presented in Eq. 1.

Fig. 6

Yield map of a dry onion (Allium cepa L.) field. Yield was estimated using the Normalized Difference Vegetation Index calculated from reflectance data acquired using a UAV mounted multispectral MicaSense Red Edge 3 sensor

$$Yield\left( t \right){\text{ }} = {\text{ }}f\left( {VI\left( {KS} \right)} \right)$$

(1)

where Yield(t) is the mass of harvest/unit area at time t, f is an empirical-defined function (i.e., a linear model) and VI is a vegetative index (i.e., Normalised Difference Vegetative Index) measured at the key phonological stage KS.

The approach, as presented in Eq. 1, has been proposed for tomatoes (Johansen et al., 2019) or potatoes (Li et al., 2020). The calibration of the f(t) model requires a database where the values of VI at KS and Yield(t) are known. Although effective, this calibration is a limitation because the model remains valid only locally on the set of data used for the calibration. Furthermore, Eq. 1 assumes that the f(t) model, once determined, is stable over time. This hypothesis of temporal stability being constraining, the collaboration between VI derived from images and specific ground yield data could recalibrate the model accounting for specific conditions of the year. The overall idea is to provide a simple yield prediction strategy, combing satellite based VI and limited within-field samples for improving yield estimation before harvest (Carrillo et al., 2016; Sun et al., 2017), while other authors have proposed the implementation of adapted sampling approaches in order to optimise the calibration of the model with a minimum number of field measurements (Miranda et al., 2018; Oger et al., 2019).

The approach, as presented in Eq. 1, assumes that there is a single KS for which yield estimation based on a vegetation index is possible. To overcome this limitation, some authors have proposed the use of time series of multispectral images to characterize crop evolution at different phenological stages. This approach has been successfully proposed by Moeckel et al., (2018) to predict the yield of eggplant, tomato, and cabbage in a phenotyping context. It has also been proposed by Sun et al., (2017) for vines with correlation coefficients between yield and VI ranging from 66 to 82%. However, the approach remains broadly the same as that presented in Eq. 1 with the difference that the f model is multivariate which also requires a significant database to perform its calibration.

Other approaches developed are based on allometric principles in which the yield is assumed proportional to the size of the plants or the trees. High spatial resolution multispectral images are used to assess characteristics of individualised plants. Sarron et al., (2018) and Uribeetxebarria et al., (2019) successfully applied this approach on mango and peach trees, respectively. They have shown that the projected crown surface of individual trees estimated from above was strongly correlated to the number of fruits or to the weight of fruits. More complete approaches make use of photogrammetry to extract 3D parameters like the height and/or the volume of individual plants or trees. Once again, yield prediction is based on empirical models that relate the volume or the area of each individual plants/trees to the yield or a yield component. In this case, the model of prediction is similar to that of Eq. 1. except that the vegetation index is replaced by a volume or surface variable of each plant. Similar assumptions (local calibration, temporal stability of the model) apply for this type of approach. Care must be taken to ensure that other covariant factors do not cause problems. For example, tree diseases affected the accuracy of yield maps generated by MacArthur et al., (2006) with an UAV and machine vision algorithms.

Remote sensing is also used to provide assessment of a yield estimation a long time before harvest. For some perennial crops, the number of flowers can be considered as an estimate of the number of fruits at harvest, provided that no climatic event will affect this number during the growing season. Given the size of the flowers, the spatial resolution of remote sensing images does not generally allow an exhaustive count of each flower, but rather an estimate of the number of flowers per unit area. This has been proposed for crops for which flowering corresponds to a significant change in reflectance at one or more wavelengths in the visible and at an early stage when the leaves are not yet developed. Counting flowers to estimate yield has been proposed on strawberries (Chen et al., 2019), peach trees (Horton et al., 2017), and apple trees (Aggelopoulou et al., 2010; Tubau Comas et al., 2019; Xiao et al., 2014). Other yield components have been estimated by remote sensing in horticulture, such as the number of living or productive plants. In this case, it is not a matter of estimating the yield directly, but rather to account for a correction when the yield is estimated at the plant level, whether by sensors or manual counts. The yield is thus expressed as the mass of harvest per plant and then multiplied by the number of productive plants per block. The estimation of the missing plants (dead or unproductive) aims at improving the yield estimation by knowing more accurately the number of productive plants per block. Associating yield to the crop biomass is mainly applied to perennial plants such as vines where a decrease in the number of productive plants may occur throughout the years of the lifespan of the block. This requirement explains why significant work has been devoted to the detection of missing plants by remote sensing with the objective of better estimating the yield (Primicerio et al., 2017; Robbez-Masson & Foltête, 2005).

Discussion

Comparison of the different approaches

Horticulture varies widely in the types of crops, the production system, the environmental situations, and the characteristics of the farms or organizations. As discussed in previous sections, this has resulted in many different methods and sensors for measuring yield.

Horticultural crops are usually marketed by weight, volume, or counts. It is ideal if the yield sensing is done on the same basis. However, the sensing may be done on a different basis for reasons such as sensor cost (Schueller et al., 1999), accessibility or non-interference with the crop (MacArthur et al., 2006), or technical feasibility. If the crop is marketed on a different basis than it is sensed, there needs to be a conversion which likely requires a calibration. Even if the sensing is done on the same basis, there may be a need for calibration to compensate for inaccuracies. For all yield sensing systems, calibration is an important component.

Yield sensing approaches can either be “deterministic” or “empirical” (Table 1.). A deterministic sensing approach will measure the quantity of interest directly. Consider, as an example, the many crops marketed by weight. For these crops, accurate weighing will determine the yield. Alternatively, if machine vision is to estimate a yield of the same crop (e.g., Gan et al., 2018), that empirical approach will have to be calibrated or otherwise converted to weight to get an appropriate yield measurement. In general, a particular deterministic sensor will be more applicable to a wider variety of commodities and situations because crop-specific calibration is not required. Continuing the same example, a weighing sensor on a conveyor or accumulating bin will be applicable to many commodities and situations. Empirical systems tend to be optimized more for particular crops and situations. Certain approaches can be either empirical or deterministic depending on the type of produce and situation. For instance, a high throughput volumetric measurement using a LiDAR system above a conveyor will be considered deterministic if all harvested produces are visible (e.g., potatoes, oranges, beets…), and would rather be considered empirical if the produce is layered (e.g., baby spinach versus baby frisee). In the later example, the baby spinach would occupy a smaller volume than baby frisee for a similar yield. It is thus important that empirical approaches proposed as yield sensing solutions be specific about what it is designed for.

Accuracy problems can occur if there is insufficient data to establish the true accuracy of the sensing method over the entire range of operation (Chan et al., 2002). Calibration requires sufficient data points and replications that adequately cover the inference space of all the yield ranges and the conditions under which the yield sensor will be used, and that there are no confounding covariants. Accuracy may also be compromised if the effects of changing conditions or covariants cannot be compensated. Yield sensor calibration has usually used least-squares regression software on computers, but neural networks and other artificial intelligence methods are becoming more popular (e.g., Kalantar et al., 2020).

Historically, yield sensors have been calibrated before use. But this does not guarantee that they will remain accurate as crops, sensors characteristics, and acquisition conditions can dynamically change. Calibration and conversion could be updated with some feedback as the sensors continue to be used to increase their accuracy and to accommodate changes. This can be done as sensing progresses in “real-time” or by post-processing the yield data. One method is to look at the total quantity accumulated at some point in the postharvest transport, processing, or packaging process and compare the amount of actual harvested crop to the cumulative yield sensed and adjusting the yield sensing calibration accordingly. Another method is to use another yield sensor simultaneously (Zhang et al., 2018). Even if the second sensor has a poorer dynamic response and is subject to more yield information smoothing and delay, it can help recalibrate the initial sensor if it is inaccurate. Horticultural yield sensing research and development has been slow to utilize Kalman filtering and other advanced techniques (e.g., Blok et al., 2019) to improve dynamic responses to changing conditions and multiple sensing techniques.

Horticultural products often have quality requirements. Some of the empirical yield sensing techniques have the advantage that they can enforce these quality requirements. For example, a machine vision yield sensing system may exclude from its measured yield those fruits or vegetables which are damaged or do not meet product specifications. In this way, the sensor will not record defective production as yield.

The Table 1 shows a comparison among the measurement principles based on their classification, complexity, readiness, capacity for generating a forecast, adaptation for onboard operations and potential for qualification of yield. Only three measurement approaches are deterministic, which suggests that calibration remains a concern for most yield monitoring principles, as it is currently the case for grain yield monitoring (Nielsen, 2020; Pennington, 2016). Most approaches require complex systems composed of either the integration of multiple components (e.g., vehicle tracking or machine vision) and/or the use of advanced technologies generating complex datasets requiring advanced computing techniques (e.g., ultra-wideband radar or machine vision). This complexity may explain why such approaches have not been commercially deployed yet, but with the rapid advance in high-performance computing systems, this may not present a barrier to adoption in the future. Nevertheless, the development stage seems to be inversely correlated to complexity and this may potentially be explained by the difficulty of generating reliable data with a complex system operating in an uncontrolled environment with variable conditions such as lighting, soil surface, pests, and temperature (Pajares et al., 2016). Another aspect to consider is the presence of the yield monitoring approach on-board the harvester or being used during the harvesting process, and the possibility for yield mapping prior to harvest. The more traditional deterministic yield monitoring approaches, such as the use of load cells or volumetric measurements, need to be used during the harvesting process. In contrast, more recent empirical approaches, such as remote sensing and machine vision, offer the possibility to be used in a step prior to harvesting and thus allow yield forecasting, which may confer an additional layer of value for the farmers (Horie et al., 1992). The last aspect considered in Table 1 is the potential of each approach for the qualification of the harvested produce. Only two approaches appeared to show potential for some aspects of produce qualification, which are volumetric measurement and machine vision (Moreda et al., 2009; Naik & Patel, 2017). In horticulture, crop qualification remains an important aspect of yield sensing because the marketable price can vary based on classification, even if the harvest weight or volume is constant. González-González et al., (2020) have developed a system to map citrus yield that shows the spatial variability of citrus categories defined by diameter across the orchard. Table 1 also shows that certain approaches such as machine vision and load cells are more versatile to work across different crop types than others that are more targeted to specific crops such as ultra-wide radar sensing for root crops.

Table 1 Comparison of the different measurement principles and relevant literature

What level of accuracy is required for each task/crop?

Previous sections have illustrated the wide variety of yield sensing principles available for horticultural crops. These tend to generate data at different spatial resolutions that are determined by the type of yield sensor used and the mode of harvesting. For mechanical harvesting, most horticultural crops are at an advantage to broad-acre crops as harvesting is typically done on a single tree (stand-alone tree orchards) or on narrow transects (2-4 m) associated with either a single row (e.g., vineyards or trellised fruit systems) or only a few rows (potatoes, market vegetables, etc.) of crop. In mechanically harvested crops, there is usually little convolution or mixing within the harvester, i.e., the crop passes quickly from the plant, across the sensor to the collection container or trailer. As such, the data from continuous yield sensors are typically associated with a point in the field. For non-continuous systems (e.g., the automatic weighing of bins in an orchard in Whitney et al., 1999) or for the weighing of hand-harvested fruits, the spatial footprint may cover several individual trees or plants and will be an area measurement. In either case, the footprint of the yield data is usually small and/or well-defined, such that the geolocation of the data is more dependent on the accuracy of the GNSS receiver used, than on yield monitor effects. This contrasts, for example, with cereal crops that often use large (e.g., 10 m) cutter bars and have long lag times for the grain to move through the harvester. It is expected that a high accuracy GNSS receiver associated to a high quality and well-calibrated sensor will give precise and accurate yield data. These data may therefore appear very noisy as the short-term stochastic variance in the production system may be captured in the data.

The required accuracy of the data will be dependent on the end objectives. Several advantages of generating crop yield maps have been listed in the introduction. For practical purposes, these different uses of a yield map can be broken down into three main categories: (1) understanding spatial variability, (2) auditing production and (3) steering decisions for site-specific management.

For understanding general spatial patterns in yield response, yield monitors do not need to be accurate or precise, provided some effective form of smoothing or interpolation is applied in the generation of the final maps, and areas of the field are only compared on a relative basis. For instance, Pelletier & Upadhyaya (1999) observed a 50% yield reduction between the 20% most productive and the 20% least productive areas of the field, thus providing an important insight to the grower solely based on relative observations. Low accuracy often becomes de facto the case in agriculture if yield monitoring systems are poorly calibrated or operated. The key assumption is that any error or bias is either constant or proportional to the yield response. The final data can only be used to identify trends, not exact values of yield. This can inform growers on macro-variability existing in their fields and raise questions about poorly drained areas, significant changes in soil texture or edge effects for instance (Grisso et al., 2005; Schumann, 2010).

Some producers use yield sensors to obtain in-field information to audit harvested tonnage against delivered tonnage. This information is typically associated with blocks or areas of a production system. In this case, accurate, but not necessarily precise, yield monitors are needed, i.e., an individual point measurement may not be accurate, but the summed total is accurate when averaged over a bin or a truckload. It is important to note that even the best yield sensors will still be subject to some error and yield sensors are not sufficient to correctly audit exact tonnages at the moment. However, they can identify significant deviations (e.g., > 5% difference) which may be associated with losses in transport or logistical operations (Taylor et al., 2016).

When the intent of the yield data is to quantify and validate the effect of management on production, then producers need yield data that is both precise and accurate. This requires high quality yield sensors, high accuracy georeferencing systems and conscientious sensor and machinery operation during the calibration and harvesting processes. Accuracy in yield data becomes important when using these maps on an actual basis as opposed to a relative basis. The literature based on grain crops suggests that trends must be aggregated on multiple years in order to provide reliable information to growers (Blackmore et al., 2003; Xiang et al., 2007). Accurate yield data helps in comparing yield maps from one year to another, which would otherwise need to be normalized and thus lose information. In addition, for crops harvested multiple times during the crop growing season (e.g., peppers, zucchini, etc.), accurate yield data become mandatory to generate a cumulative yield map. The use of yield maps by the majority of grain growers for their decision making remains a challenge of precision agriculture to date (Bramley & Ouzman, 2019) and, if suggested approaches to on-farm experimentation are to be widely adopted (e.g. Bullock et al., 2020; Marchant et al., 2019), then yield monitors must generate yield data at a quality that generates confidence in making very targeted crop-specific decisions (and not just an assessment of spatial trends).

Current trends and future perspectives

As highlighted by the multiple advantages of possessing a yield map in the introduction, the demand for reliable and cost-effective yield monitoring system for horticultural crops will continue to grow. Indeed, horticulture crop growers understand the benefits of the data revolution in agriculture for productivity and environmental stewardship. The consumer demand for clean food along with stricter government regulations will maintain pressure on growers for more efficient use of resources (i.e., soil, water, fertilizers, and pesticides), which will be linked to the adoption of precision agriculture for mechanized farms (Schneider, 2017; Zhao et al., 2018). Yield maps are key for the optimization of inputs because they provide a spatialized result of the operations, a feedback loop that is important for spatial optimization of input use (Basso & Antle, 2020).

Calibration of yield monitoring systems has been a challenge for grain crop yield sensing and will remain a challenge for horticultural crop yield sensing as well (Nielsen, 2020). One aspect that will be investigated and may become more attractive for yield monitoring technologies is autonomous calibration. A self-calibrating system would increase reliability of yield data and thus improve usability for decision making. Technologies for autonomous calibration have been studied for grain crop yield monitoring (Beck & Pickett, 2006; Morgan, 2020). Based on the experience learned from grain crop yield sensing, developers of yield sensing technologies for horticultural crops may consider including self-calibration by design in their systems.

Agricultural robots may disrupt the development of yield sensing technologies. Agricultural robots will bring a new perspective by being close to the crop and operating at a relatively slower speed than current farm machinery. This will enable the use of tailored machine vision systems capable of collecting meaningful crop data enabling yield forecasting, yield mapping and selective harvest planning (Lowenberg-DeBoer et al., 2020; Underwood et al., 2016).

Another aspect of yield monitoring that will likely gain traction is sensor fusion. The advantages of sensor fusion include: reduced uncertainty (same information from multiple sources), better accuracy and reliability (bring redundancy in the system), improved resolution (combination of multiple measurements), and reduced complexity (dedicated sensors rather than a very complex sensing approach). Using multiple sensors increases the variety of data that can advantageously feed artificial intelligence algorithms (Misra et al., 2020). The addition of signals often reduces the error by increasing redundancy. The combination of sensing technologies will also simplify yield quantification and qualification as opposed to the use of one sensing approach. Based on existing literature, one can envision a yield monitoring system combining load cells to measure the harvested grape weight on-the-go along with florescence sensors measuring grape quality at the same time (Baluja et al., 2012). As mentioned previously, sensor fusion also presents options for autonomous calibration across multiple sensors (Pereira et al., 2016).

Machine vision is gaining momentum and new applications will continue to emerge. Machine vision-based solutions have already been investigated for all categories of horticultural crops covered in this study (Table 1), which shows the potential and the interest for this approach. Machine vision is the only approach offering the potential for both quantification and qualification of crop yield (González-González et al., 2020). Because of the relative simplicity of sensor deployment (i.e., installing cameras) and advances in deep learning implementation and powerful computing, machine vision is the approach showing the most potential for yield sensing of horticultural crops. However, it remains an empirical approach and limited by occlusion issues (Schueller, 2021), thus its combination with deterministic approaches, such as load cells will probability be basis of the most robust approaches to come.

Conclusions

Yield sensing and the availability of yield map present several advantages and opportunities for horticultural crop growers, thus justifying the continued development of monitoring approaches to generate reliable yield maps. The advent of digital agriculture and precision agriculture, which relies on crop management, environment, and production data to optimize the use of resources is highly dependent on yield maps and reliable yield monitoring technologies will enable horticultural crop production to fully embrace this new paradigm. The few commercial yield sensing technologies that exist for horticultural crops tend to be based on the load cell principle measuring yield mass on conveyor belt on the harvester on-the-go. Direct yield measurement can also be done on a volumetric basis or by counting production units (i.e., single fruit, bags, bins or boxes). The other approaches tested in the literature involve indirect yield measurement, which can be done by capacitance, vehicle tracking, machine vision, radiometry, remote sensing or by using RFID for yield tracking. Yield monitoring approaches can be classified as deterministic (i.e., measuring the quantity of interest) or empirical (i.e., establishing correlation between the sensed parameter and the quantity of interest). The deterministic approaches have the advantage of showing more robustness and being simpler while empirical approaches require more complexity due to conversion and are more adaptable to different contexts. The choice of the right yield monitoring approach is linked to the different context of each crop and should consider multiple factors such as adaptability, reliability, precision, and accuracy. The great diversity of products and organizations in horticulture explains the high diversity of measurement principles observed in the literature. Given this diversity, there will not be one measurement technology suitable for horticulture, but a range of suitable technologies adapted to each context and each product. This context will continue to call for original projects and research questions in the future. In general, despite the numerous papers published on yield monitoring approaches for horticultural crops, several problems remain before commercialisation. For instance, capacitance and volumetric sensing have received little attention as well as the fusion of multiple approaches. Machine vision and sensor fusion appear as promising approaches for monitoring crop yield on-the-go. The digital agriculture revolution for horticultural crops remains contingent upon the development of reliable and ubiquitous yield monitoring systems.