Introduction

Satellite imagery has become a promising and versatile source for global agricultural monitoring in recent years (Kussul et al., 2015, 2020; Nandibewoor et al., 2015; Nguyen et al., 2020; Serra & Pons, 2008; Wu et al., 2015; Zhang et al., 2020). The use of image times series from the same scene at different periods has driven the interest in regularly monitoring crops at the Earth’s surface (Valero et al., 2015). However, the mapping of crop areas and the identification of crops through the analysis of multitemporal satellite imagery are not yet easy problems in remote sensing. For example, obtaining a cloud-gap-free image series over an extended period is a challenging task that limits the ability to characterize crop areas. Also, due to the strong phenological variability on a global scale, crop identification and mapping have become complicated tasks.

In the last two decades, the application of remote sensing has significantly progressed and has widely been applied to classify crops, estimate and monitor crop area, and crop yield predictions (Wójtowicz et al., 2010). Crop classification is the base of most remote-sensing applications in agriculture (Zhang et al., 2022). The classification based on mono-temporal image limits the accuracy of land use classification as different land covers may have the same spectrum at a given time instance (Odenweller & Johnson, 1984). To overcome this problem, multi-temporal images are analyzed for classification based on time series which captures the crop’s biophysical features. Spectral signatures as a time series can reveal changes in phenology and inform about the health of a crop over its life cycle (Foerster et al., 2012).

Processing multitemporal satellite images to generate a data time series has now become convenient with Google Earth Engine (Lindsay et al., 2022; Shelestov et al., 2017). Many studies have incorporated time series satellite imagery for crop classification and crop health detection e.g. (Dong & Xiao, 2016; Dong et al., 2016; Hao et al., 2014; Liu et al., 2020; Ozdarici & Turker, 2007; Setiawan et al., 2022; Turker & Arikan, 2005; Wang et al., 2020; You & Dong, 2020). However, remote sensing vegetation index products contain a lot of noise caused by precipitation, cloud cover, human influence, and the sensor itself in the data acquisition and process stages, and these factors can be collectively called random factors or residual parts (Bai et al., 2020).

Yu et al. (2021) allude to the advantages of high-resolution NDVI time series in revealing detailed information on vegetation at a regional scale. However, residual parts and long revisit frequency of certain satellites introduce large gaps in the NDVI time series data and limits their application in related studies (Julien & Sobrino, 2019; Þórðarson et al., 2021). Thus, an effective reconstruction method is required for the generation of continuous NDVI time series (Chen et al., 2021; Chu et al., 2021; Yu et al., 2021). Another issue is the presence of noise within the image due to coarse scale, clouds, contaminated pixels, or aggregation method which may result in spikes or disturbance in the time series. To address this issue smoothening filters are applied in remote sensing. The field validation of the vegetative index (based on multi-temporal satellite imagery) improves the accuracy while there are some new techniques that can be used to fill the gaps in time series and reconstruct the vegetative index trajectory e.g. development of a hybrid piecewise logistic model (Zhang, 2015), application of Savitzky–Golay filter for smoothening and reduction of noise (Chen et al., 2004).

Pakistan’s economy heavily depends on contributions by the agriculture sector (featuring 23 Mha cultivated land resources and 37% of the country’s labor force). The recent Economic Survey (GoP, 2023) reports growth in the agriculture sector by 4.4% during 2021–22 mainly due to high production of crops. Wheat is the most favorite crop in Pakistan which is grown on 39% of cultivated land of the country. Saeed (2022) argues that whether wheat is a suitably profitable crop or not, Pakistan’s farmers cannot quite resist growing it due to its importance in national food demands and government support policies since the 1960s. Every year targets for wheat production are set and provincial governments put in resources and efforts to enhance wheat productivity and achieve targets. Crop monitoring hence is crucial to keep track of provincial targets, assessment of crop health, and estimation of yield. On a national scale expensive decisions are based on crop monitoring e.g., import orders of agricultural commodities; introduction of interventions for productivity enhancement; and damage assessment in case of a natural disaster.

This paper is therefore focused on crop phenology monitoring as an application of time-series remote sensing data. The signatures of crop phenology are derived based on the normalized difference vegetation index (NDVI) as a measure of the photosynthetic activity of vegetative land cover. This study also investigates the use of on-ground imaging devices for validation of NDVI from satellite products; and evaluates the utility of smoothing filters and techniques for the regeneration of satellite time series.

The specific objectives of this paper are to:

  1. i.

    Acquire and process multispectral images captured by Sentinel (L121-L192) satellite for four experimental fields of and apply gap filling and smoothening filters to prepare crop phenological signatures using GEE platform.

  2. ii.

    Conduct a field phenotyping experiment to assess growth dynamics of various crops simultaneously in high temporal (daily) resolution using phenocam as a high-throughput approach.

  3. iii.

    Validate low resolution phenological signatures (based on Sentinel (L121-L192) data) of crops using high resolution phenological signatures (based on phenocams data) and select crop for sensitivity analysis and classification based on this validation.

  4. iv.

    Analyze sensitivity of Sentinel (L121-L192) data for wheat crop at different phenological stages to various parameters of selected smoothening filter and gap filling method and comparing it with high resolution (phenocam) data as a reference.

  5. v.

    Classify wheat crops over a wider area (including the experimental fields) using Sentinel (L121-L192) data by incorporating Random Forest Machine algorithm in GEE.

Data and materials

Ground validation of NDVI is performed on a small agricultural farm maintained by the Pakistan Council of Research in Water Resources (PCRWR) which is a public sector research institute in Pakistan. The farm is located in the Sargodha district (area of 5845 km2) of the Punjab province. Sargodha is located at the northwestern part of the Indo-Gangetic plains. Its soil is of alluvium origin carried by the Chanab and Jhelum rivers making it a part of interfluvial plane of the two rivers, locally named Chaj Doab. The northern part of the district is surrounded by a salt range. The old alluvium is a composite of calcareous and silt clay. The district is famous for its citrus orchards. Sargodha districts belongs to ‘Mixed Punjab’ agro-climatic zone (out of total 9 zones) (Malik et al., 2012) where popular crops, besides fruit orchards, are wheat, sugarcane and rice. With respect to agro-ecology, Pakistan is categorized into 10 distinct zones. Sargodha is in the zone of Northern Irrigated Plain. Ahmad et al. (2019) provided detailed characteristics of agro-ecological zones in Punjab province and provided crop suitability maps for these zones.

The climate of Sargodha is arid characterized by negative moisture balances, i.e., potential evapotranspiration rates permanently exceed precipitation rates. Summers are hot and longer than mild winters. The maximum temperature reaches up to 50 °C during summer with May–July being the hottest months. The mean annual precipitation in Sargodha is around 440 mm (based on data from 2002 to 2020) mostly concentrated in the months of July and August (monsoon season). Figure 1a shows the location map of Sargodha district and the selected fields in PCRWR’s agricultural farm. A photograph of field cameras installed in the farm is shown in Fig. 1d.

Fig. 1
figure 1

a Map of Pakistan highlighting Sargodha district; b Land classification within Sargodha district; c experimental fields; d picture of phenocams installed on a pole

Full size image

Methods

The timing of periodic events in plants, like budburst or flowering, is generally referred to as plant phenology. Phenological signatures of crop can be captured by analyzing phenometrics over time. (Sunoj et al., 2016) explain two distinguished types of phenometrics widely used in literature: (i) Color metrics —The calibrated images are used to measure quantitative information. Each color channel of the captured color image (RGB) consists of pixel intensities ranging from 0 to 255. A quantitative measure (color indices) can be extracted from any color image with the pixel intensity values of each layer; (ii) Radiometric measures—are not extracted from RGB images but they are recorded using a spectroradiometer at specific wavelengths e.g. Normalized Difference Vegetation Index (NDVI) and its variants like Enhanced NDVI, Green NDVI, Meris Terrestrial Chlorophyll Index (MTCI), etc.

NDVI is a numerical indicator used to characterize the greenness of live vegetation (Rouse et al., 1974; Tucker, 1979). NDVI has been used in numerous studies (e.g. (Hufkens et al., 2012; Inoue et al., 2015; Migliavacca et al., 2011) of plant extent and magnitude using satellite imagery (Leon et al., 2012). This study has therefore used NDVI as a representative vegetative index to study crop phenological signatures.

To achieve the specific objectives of this paper the dataset consisted of two types of imagery i.e. (i) captured by satellite, and (ii) phenocams.

The first set of data in this study consists of multispectral images captured by the Sentinel-2 satellites i.e. Sentinel (L121-L192). The Sentinel-2 Multi-Spectral Instrument (MSI) has medium resolution and consists of 13 spectral bands with free access to source data (Zhao et al., 2024). Sentinel-2 is the first mission that provides free publicly accessible data at 10–60 m resolution (10 m in four spectral bands) every five days with global coverage (Drusch et al., 2012).

Sentinel (L121-L192) data was acquired temporally. This is because the cropping calendar for various crops are not uniform across Pakistan. Also crops have different appearances dependent on their growth and cultivation period. Raza et al. (2022) argues that some of the crops might have almost indistinguishable appearances in early growth period making one image insufficient for crop identification. It is therefore more useful to analyze time series of satellite images keeping in view peak growth stage of a crop and then use appropriate image(s) to differentiate the crops from each other.

The second data set is acquired using phenocams. Vrieling et al. (2018) report that trained personnels or volunteers used to monitor phenology with visual observation by physically visiting fields. Fixed-position digital cameras that photograph of plants at high frequency can minimize or eliminate physical field presence. This so-called digital repeat photography has been implemented in various networks in Switzerland (Aasen et al., 2020; Liu et al., 2022a, 2022b), United States (Longchamps & Philpot, 2023; Sunoj et al., 2016) Australia (Moore et al., 2016), Japan (Inoue et al., 2015), India (Hufkens et al., 2019). Besides identifying discrete events, digital repeat photography makes possible to analyze temporal changes in red, and near infra-red (NIR) channels allowing a more continuous tracking of normalized difference vegetation index (NDVI). According to Aasen et al. (2020) the technique of digital repeat photography that continuously capture images of a given area with an RGB or near-infrared enabled cameras (also called Phenocams) has been used for more than a decade to estimate phenology (Ahrends et al., 2009; Graham et al., 2010; Hufkens et al., 2019; Ide & Oguma, 2010; Kurc & Benton, 2010; Migliavacca et al., 2011; Penuelas et al., 2009; Richardson, 2019). These phenocams can deliver very high spatial resolution data at high temporal frequency. Keeping in view these capabilities of the phenocams, this study has made use of phenocams to acquire high resolution daily NDVI data in the selected experimental fields. This data is thereby used as a reference when phenological signatures from Sentinel (L121-L192) images are compared and sensitivity analysis. The details of phenocams, experimental setup and sensitivity analysis is provided in subsequent sub-sections.

Phenocams, field setup, and image processing

Phenocams used in this study are four field-grade cameras (colloquially referred to as CCFC cameras after the name given by the manufacturer Campbell Scientific Inc. USA) capable of capturing NDVI and RGB images. Aasen et al. (2020) state that usually, phenocams are mounted on towers or lookouts and view the canopy horizontally or obliquely to record the objects within their field of view several times a day. In the case of this study all four CCFC cameras were installed on a single pole at the central point of four experimental fields each measuring 30 × 30 m in dimension. The field boundaries were set up to overlap with the grid cells of satellite images of 30 m resolution. Each camera was installed on a small cantilever pipe fixed on a pole at a height of 20 ft from the ground. The cameras were positioned to look at the four adjacent fields obliquely and programmed to take one picture daily at 10:00 am to synchronize the time when satellite images are taken. The meta data of satellite images confirms that the images are taken at 0600 UMT which reasonably corresponds to the local time stamp of phenocam images.

The CCFC cameras have a focal length of 4.70 to 64.6 mm and field of view ranging from 4° to 67.3°. The camera comes with two IR LEDs and is capable to capture NDVI images when programmed to do so. The visible spectrums of NDVI images are: 550 to 700 nm in the visible red spectrum, and 730 to 1000 nm in near infrared (The CCFC camera measures this range only when the IR filter is removed during NDVI image capture.) The CCFC cameras take a NDVI image and stores in the internal memory in JPG format. All NDVI images are stored with a color bar at the bottom of the image for reference. The colors in the bar represent NDVI values ranging from − 1.0 to 1.0 (More details are available at manufacturer’s website https://www.campbellsci.es/news-ccfc-firmware-ndvi). The CCFC cameras did not have a feature to extract numeric values of NDVI for every pixel of the image. Therefore, a Python script was developed to calculate NDVI for every image captured by CCFC camera. This code calculates NDVI pixel by pixel with reference to the color bar and hence takes much processing time. An average computer takes around 7 h to process one NDVI image and calculate the aggregated value of NDVI. In Feb 2023, Campbell Scientific updated the CCFC firmware which makes it possible to extract pixelated NDVI values in CSV format. However, during the study period this option was not available.

The dimension of the images was set width = 2592 pixels and height = 1944 pixels, the resolution of the images was 96 dpi (both horizontally and vertically), and the bit depth was 24. The cameras were not installed at any specific angle, but their focus was set to capture a representative image of the field. The image collection with CCFC cameras started on 26 May 2021. The images up to 11 Nov 2022 were collected and used for analysis in this paper. This makes the validation period 534 days and the maximum possible 534 daily images available with each camera. Few images were not captured though when there was not enough power for cameras. This usually happened during long spells of fog or cloudy days. Images from CCFC cameras have been taken during three consecutive crop seasons i.e., two summer crop seasons (locally called Kharif) of 2021 and 2022; and one winter crop season (locally called Rabi) of 2021–22.

Figure 1b shows the land classification map of Sargodha district location of Sargodha district showing that majority of the land is categorized as crop land. Figure 1b is a Google Earth image of the selected field site while Fig. 1d shows a picture of CCFC cameras installed on a pole. The cameras were powered by a solar panel and backup battery and programmed to take a picture and go to sleep mode to conserve power. Crop rotation in the selected fields was not controlled i.e., the farm manager routinely decided about crop type and agronomic practices. Table 1 shows the crops grown in the selected fields along with the sowing and harvesting dates. The table also shows the number of images available from each camera and the paired data size i.e., the number of instances where both field and satellite image for a selected field is available. The paired data ranges from 38 to 52 which is a reasonable size for the comparison. The table also shows that in some cases a single crop is grown in multiple selected fields in one season and in other cases, a crop is repeated in the same selected field in multiple seasons. In both cases, when paired data is organized by crop the data size is enhanced.

Table 1 Experimental design and field images during the validation period
Full size table

Satellite imagery and analysis

In this study, temporal Sentinel-2 (more specifically i.e. Sentinel (L121-L192)) satellite images were utilized for the time series analysis and capturing phenological signatures of various crops particularly the winter wheat. Sentinel-2 is a constellation of Earth observation satellites developed by the European Space Agency (ESA) as part of the Copernicus program. The satellites are equipped with a multispectral imaging instrument that provides optical imagery of the Earth’s surface. One of the unique aspects of the Sentinel program is its open data policy. The imagery captured by Sentinel-2 is freely available to the public, promoting global collaboration and enabling researchers, scientists, and organizations to access and use the data for various applications. Time series analysis and image classification were performed in the Google Earth Engine cloud computing platform. After importing the Sentinel-2 imageries for the region of interest into google earth engine (GEE), the data cleansing steps (including cloud masking) were also performed as applied by Zhao et al. (2024). These steps and removal of noise and artifacts were performed to enhance the quality and usability of the imagery.

Time series analysis of NDVI based on Sentinel-2 images of the experimental fields helped in understanding the phenological variations of the cultivated crops in a time bracket from 15 Apr 2021 to 05 Mar 2023. Images with less than 10% cloud cover were selected for analysis and any cirrus, shadow, and snow pixels were removed. The average NDVI from satellite and CCFC images was compared statistically. Ordinary least squares (OLS) regression was applied to paired data sets and the coefficient of determination R2 is used as a statistical indicator. The coefficient of determination explains the strength of a linear relationship between two quantitative variables. The R2 values 1 or 0 would indicate the regression line represents all or none of the data, respectively. A higher coefficient is an indicator of a better goodness of fit for the observations.

Smoothening filters

Many smoothening filters and gap-filling techniques can be used to reconstruct NDVI time series e.g. Mean-value iteration filter, the modified best index slope extraction, Fourier Transform, Savitzky-Golay (SG) filter, Asymmetric Gaussian function fitting, and Harmonic analysis of time series (Hang-Yan, 2010). However, the most widely used and effective method has been SG Filter (see for example; Bojanowski et al., 2009; Chen et al., 2004; Kim et al., 2014; Hang-Yan, 2010; Liu et al., 2022a, 2022b). The original method proposed by (Savitzky & Golay, 1964) was based on smoothening of data by simplified least square procedure if the data fulfills two conditions: (i) the data points must be at a fixed, uniform interval in the chosen abscissa; (ii) the curve formed by graphing must be continuous. This condition is not fulfilled in the case of NDVI time series which is acquired by processing satellite imagery in this paper. Ideally, the satellite images (from a particular product) for a given area are available at a fixed interval (5 days in our case). However, the GEE algorithm used in this study applies some filters to reduce noise from clouds and shadows, therefore the resultant time series is not uniform. Savitzky and Golay (1964) had realized this issue and proposed an alternative method to smooth fluctuations by using a moving average.

The SG filter can be understood as a weighted moving average filter with weighting given as a polynomial of a certain degree. Chen et al. (2004) explain that the weight coefficients when applied to a signal, perform a polynomial least-squares fit within the filter window. Their observation is that two parameters are important when the SG filter is applied to NDVI time-series smoothing. The first parameter is the half-width of the smoothing window. A larger window produces smoother results at the expense of flattening sharp peaks. The second parameter is an integer that specifies the order of the smoothing polynomial, a smaller value of polynomial order produces a smoother result but may introduce bias while a higher value will reduce the filter bias and may give a noisier result by overfitting.

In this paper, a sensitivity anaysis is performed to test the observation of Chen et al. (2004) by considering parameters of fixed uniform interval, polynomial degree, and the bounds of the consecutive data. The daily NDVI time series based on phenocams is used as a reference for statistical comparison of phenological trends. The GEE code used for analysis explained in this section is provided at https://code.earthengine.google.com/fa78c9c7df243093b7f46b765e3be437.

Crop classification

The Random Forest (RF) algorithm has been widely used for classification of crops using satellite imagery e.g. (Akbari et al., 2020; Nasrallah et al., 2018; Ok et al., 2012; Dixon et al. 2021; Zhang & Yang, 2020). The RF algorithm was first introduced by Breiman (2001). According to Archer and Kimes (2008) RF is an advanced version of bagging ensemble classification method with randomness added to it. RF classifier is a collection of tree-structured classifiers node and no pruning or stopping rule. Ok et al. (2012) explain that RF splits each node using the best among a subset of predictors randomly chosen at that node. In this way a new training data set is created from the original data set with replacement. A tree is thus grown by using a bootstrapped sample from the original learning sample (Archer & Kimes, 2008; Breiman, 2001). RF fits many classification trees to a data set and then combines the predictions from all the trees (Cutler et al., 2007). This strategy makes RF perform much accurate, fast and robust against overfitting than other supervised machine learning methods (Cutler et al., 2007; Jhonnerie et al., 2015; Ok et al., 2012; Svetnik et al., 2003).

In RF algorithm, two essential parameters to be defined are the number of trees growing and number of variables used to split each node. Probst et al. (2019) present an overview of hyperparameters tuning strategies and argue that the number of trees in a forest is a parameter that is not tunable in the classical sense but should be set sufficiently high (Díaz-Uriarte & Alvarez De Andrés, 2006; Oshiro et al., 2012; Scornet, 2017). The algorithm starts with a selection of many bootstrap samples from data. In a typical bootstrapped sample, generally 1/3 of data is not used to grow the tree, also called out-of-bag (OOB) data, but used to test the error of the predictions (Archer & Kimes, 2008; Ok et al., 2012).

Phenological signature of wheat (based on NDVI calculated from satellite and phenocam images) in the experimental field provided a reference to identify wheat in a satellite imagery of Sargodha district. In this respect, the most suitable Sentinel (L121-L192) image to be used for wheat identification was found to be of 4 Feb 2023 after some iterations. For satellite image classification, three features were defined i.e., wheat, other crops (fodder or other winter crops), and other lands (including barren land, water bodies and settlements). Using random sampling technique, a sample of 612 fields (pixels) were taken from the selected satellite image of Sargodha district. Out of total 612 samples, wheat samples accounted for 219 while 172 samples of other crops and 221 samples of other land were identified.

Random Forest (RF) machine leaning (ML) algorithm was used in GEE as a classifier considering 50 number of trees through hyperparameter tuning method. The RF classifier has been used as a supervised classifier. It was trained by considering 50 number of trees; while the predictors used were the spectral bands of the Sentinel (L121-L192) image. The image was trained using 80% of the bootstrapped samples, while the remaining 20% out-of-bag samples were used to assess the accuracy of the classified image. The statistical report included the accuracies of the user, producer, and overall, as well as the kappa value through error matrix method. The wheat area reported by the Agriculture Department for 2022 was compared to the area estimated through satellite imagery.

The accuracy of the classification has been assessed by the overall accuracy (OA) (Rwanga & Ndambuki, 2017; Story, 1986; Zhang et al., 2022) and Kappa score (Brennan & Prediger, 1981; Cohen, 1960). The GEE code used in this paper for crop classification is provided at https://code.earthengine.google.com/1d16a193b2186122417a6e9441e164b7.

Results and discussion

Comparison of NDVI from satellite and CCFC images

Figure 2 shows NDVI variation over time in Field-1. The field camera CCFC 1304 was used to take pictures in this field. During both Kharif seasons, maize was cultivated in this field while wheat was cultivated during the Rabi season. Maize is a rather short-spanned crop and is sown when monsoon downpours are received in Pakistan. During this time cloudy weather often reduces the likelihood of getting good-quality satellite data. Figure 2 shows that in Kharif 2021 fewer satellite images are available and hence the paired data is low in number.

Fig. 2
figure 2

NDVI based on satellite and field camera in Field-1

Full size image

Wheat was cultivated in Field-1 during the Rabi season. Again, the satellite images are very low in number up to late in the crop season. This is mainly because fog spells are common at this time of the year which affect satellite data acquisition. In some instances, though, the satellite image is available, but the field camera could not take the picture during long fog spells as the solar-powered battery drained out completely.

Overall, there were 40 pairs of NDVI data (i.e. when both Sentinel (L121-L192) and CCFC image was available) available in Field-1 as shown in Fig. 2. However, these 40 pairs are not necessarily captured during the lifespan of the crops. Only 19 (Maize: 2; Wheat: 8; Maize: 9) out of the 40 pairs are available while crops are in the field and the remaining 22 pairs are available when the field remains fallow and do not give much insight into crop phenological process. This information is provided in the table below Fig. 2.

In Fig. 2 NDVI time series based on images from field camera (colloquially called NDVIF) and satellite (colloquially called NDVIS) are plotted. The orange high-low lines show the inter-difference of NDVI when NDVIF is greater than NDVIS, conversely, the high-low lines are shown in blue. Mostly the NDVIF is higher than NDVIS and the fewer cases when NDVIS has a higher value are when the field is fallow. OLS regression is also performed by considering NDVIF independent variable NDVIS as the dependent variable. The NDVI from the two sources does not correlate well showing R2 = 0.207 which is very weak. The OLS regression series in Fig. 2 explains this weak correlation quite visually. The weak correlation of pooled data in Field 1 begs the question of how NDVI values correlate by filtering data pairs for individual crops. The table below Fig. 2 shows the result of the correlation coefficient, slope, and intercept of the regression equation when NDVI pairs for each crop are considered separately. There were only two NDVI pairs for Maize in Kharif 2021 and hence the regression does not make any sense. NDVI pairs for wheat correlate very well R2 = 0.917 suggesting that the OLS regression can be used confidently to correct the NDVIS based on NDVIF. The correlation for maize crops in Kharif 2021 is extremely weak although the paired data is reasonably large (n = 9). This suggests that NDVIS does not precisely capture the NDVI signatures of maize crop but surprisingly, when NDVI data for two maize crops is pooled, correlation improves substantially and R2 improves to 0.787.

Figure 3 details NDVI in Field 2 where CCFC 1304 was used to take pictures. In this field, the cropping pattern has been rice–wheat-rice. In Pakistan, this cropping pattern is followed in a huge proportion of agricultural land referred to as the rice–wheat agro-climatic zone (see Bhatti et al., 2016). Rice crops do not strictly follow the boundaries of crop seasons i.e., Rabi and Kharif—rice is sown late in Kharif, and its harvesting trespasses into early Rabi. In Field 2 the data pairs are less than Field 1 but the data available during the crop life period are much better (21 out of total 38 NDVI pairs correspond to the period with a crop in the field). A comparison of NDVI values from both sources reveals a trend that NDVIF is higher than NDVIS. This pattern is more visible in Field 2 than in Field 1 because a larger proportion of data corresponds to a cropping period.

Fig. 3
figure 3

NDVI based on satellite and field camera in Field-2

Full size image

The results of the regression analysis are given in the table below Fig. 3. Overall, paired NDVI data correlates reasonably (R2 = 0.629; n = 38). Dissecting the paired data by crops provides further insight into the relationship between satellite and field NDVI. In the case of rice, the R2 is weak (0.120) for the 2021 crop but improves (0.511) for the 2022 crop. If we pool all data for rice the correlation weakens and R2 reduces to 0.098. NDVI for wheat on the other hand shows a lower R2 = 0.606 which is better than rice but less than that for wheat in Field 1.

In Field 3 cropping pattern has been the same as that of Field 1 i.e., maize-wheat–maize. The NDVI pairs during the study period are however 32 percent larger in Field 3 than in Field 1. Moreover, NDVI pairs corresponding to the cropping period are also better (32 out of a total of 53)-in Field 3 as shown in Fig. 4. The pattern of higher NDVIF than NDVIS is also visible in Fig. 4. The correlation coefficient remained at 0.663 for maize in 2021 which dropped to 0.174 for maize in 2022. Pooled NDVI data for maize in Field 3 again resulted in a weak correlation (R2 = 0.195, n = 17). Wheat was cultivated in Field 3 during the Rabi season during which 15 pairs of NDVI data from satellite and field cameras were available. The paired data moderately correlated R2 = 0.552.

Fig. 4
figure 4

NDVI based on satellite and field camera in Field-3

Full size image

In Field 4 a single crop (i.e., sugarcane) was grown over the study period. The total number of NDVI pairs in Field 4 has been 50 out of which 34 correspond to the cropping time of sugarcane. Satellite NDVI was unavailable from the middle of December to early March due to fog as revealed in Fig. 5. The correlation between NDVIS and NDVIF is weak R2 = 0.304.

Fig. 5
figure 5

NDVI based on satellite and field camera in Field-4

Full size image

Based on the results presented in Figs. 2, 3, 4, 5 we can draw some inferences about individual crops. Maize is cultivated in two fields and with seasonal repetition. In the case of both fields as well as repetitions, the correlation between satellite and field based NDVI has been weak. If we pool all NDVI pairs for maize, irrespective of field and season, and apply regression it still shows a weak relationship (R2 = 0.335, n = 28). Rice is cultivated in only one field with a seasonal repetition. The correlation coefficient has been weak when corresponding data is pooled. Sugarcane is cultivated in only one field without repetition and showed a weak correlation. Wheat is the only crop that is cultivated in three fields and shows a good correlation between NDVI paired data in all combinations. If all 35 NDVI data pairs for wheat are pooled a reasonable correlation coefficient exists R2 = 0.505 while the R2 remained 0.917, 0.606, and 0.552 in individual Fields 1, 2 and 3 respectively.

Smoothening filters

Figure 6a–d shows NDVIS time series for individual fields. The figure also shows two smoothing/gap-filling filters i.e., moving average and Savitzky–Golay methods. NDVIS time series (green line) shows spikes in the case of all individual Fields. More pronounced spikes are visible during Kharif 2022. Fields 1 and 2, in particular, show distortion in Kharif 2022 when maize was cultivated in these fields.

Fig. 6
figure 6

Application of smoothening and gap-filling techniques to NDVIS time series for experimental fields; a Field 1; b Field 2; c Field 3; d Field 4 

Full size image

When the moving average filter (red line) is applied, it smoothens the time series quite efficiently but takes the same path as the NDVI series. The Savitzky–Golay (SG) method on the other hand takes an independent path which is representative of the NDVIS series. The SG Smoothening series in Fig. 6a, b, c, and d reasonably imitate the NDVIS and fills the gaps in data points as well.

Wheat phenology

In previous sections, we had only considered those field camera images where a corresponding satellite image was available (i.e. paired NDVI data). However, the field camera was taking images on a daily frequency. Field-wise analysis of NDVI paired data informs that NDVI from the two sources shows a reasonable correlation for wheat crops and hence should be explored further. To this end, we have analyzed all 136 available images from CCFC 1068 in Field 3. Figure 7 details the NDVI signature of wheat crops in various phenological phases. The vertical dashed lines (emergence, tillering, flowering, maturity) divide the cropping period (sowing to harvest) into 5 phenological phases as denoted by numerals in shaded boxes in Fig. 7. The NDVIS is also plotted based on 21 satellite images.

Fig. 7
figure 7

Sensitivity of SG Filter NDVI to the plynomial order

Full size image

Keeping in view the definition of SG filter and the observations of Chen et al. (2004) (elaborated in Sect. “Smoothening filter”) it is worth comparing the SG filter results with NDVIF at a daily frequency as a reference. The SG filter in the GE code used in this study, when applied to NDVIS data, first fills the data gaps to transform the time series into consecutive NDVIS data where the data points are at a fixed and uniform interval. We have also tried to test whether SG smoothening filter is sensitive to the fixed uniform interval, polynomial degree, and the bounds of the consecutive data.

To investigate the sensitivity to the bounds of consecutive data, SG smoothening filter is applied to NDVIS time series of varying lengths but it did not make any obvious difference.

Figure 7 shows the results of NDVIS when SG Filter is applied (collequillay denoted by NDVISG) using different polynomial order but keeping the uniform interval of 5 days. The interval of 5 days is selected because it represents the uniform interval at which the satellite images are available. The figure shows the shape of the resultant NDVISG series is sensitive to the polynomial order. The smaller polynomial order yields a flatter shape of the NDVISG series. Table 2 presents the R2 between the NDVIS, NDVIMA (NDVI series when 5-day moving average is applied to NDVIS series), and NDVISG time series taking NDVIF as a reference. The highest value of R2 is observed when the polynomial order was set to 4 in case of NDVISG. These iterations show the same results as noted by Chen et al. (2004) that polynomials of smaller order polynomial fitting produce smoother results and higher order polynomial fitting reduces filter bias.

Table 2 Correlation of field camera-based NDVI with satellite-based NDVI using smoothing and gap-filling methods for various development stages of winter wheat
Full size table

In Fig. 7, a polynomial trendline of order 4 is fitted to the NDVIMA series. Similarly, a polynomial trendline is also fitted to the NDVIF series. A comparison of these two trendlines shows remarkable similarity in shape. It is noted in Table 2 that the highest correlation exists between the moving average series and NDVIF i.e. R2 = 0.94. This begs the question Is there huge merit in applying SG Filter when NDVIMA series and its polynomial trend line produces better results than NDVISG series? Application of SG filter is relatively more complex than calculating moving averages and fitting polynomials with commonly used computational software (e.g. MS Excel). Therefore an informed decision should be made while selecting a smoothing and gap-filling method.

Figure 8 shows the results when the interval was varied (5, 10, 15, and 20 days) by setting polynomial order to 3 and 4 respectively for NDVISG series. The figure shows that the variation in interval did not significantly affect the shape of the NDVISG series. At large intervals, fewer points were available and missing continuous data tended to cause some filters to collapse. The filter collapsed at intervals larger than 20 days, even the filter collapsed at the 20-day interval when the polynomial order was set to 4. These results resemble the findings of Liu et al., (2022a, 2022b) who compared six filters to reconstruct NDVI in the Yangtze River Basin, between December 2018 and December 2019 and found that missing continuous data tended to cause some filters to collapse and could not effectively repair such influence when the duration of the time-series gap was larger than 20 days. Overall-fitting reconstruction method, the reconstruction time series could restore the phenological features to a certain extent if the duration of the gap was < 30 days. However, as the duration increased, the reconstruction time series exhibited obvious fluctuations and tended to underestimate the NDVI values. Here it is also important to note that the noise in NDVIS due to clouds and shadows has already been minimized in our GEE algorithm by applying filters therefore the resultant NDVIS exhibits less noisy data than the raw NDVI from satellite sources. Whereas Liu et al., (2022a, 2022b) used raw NDVI to compare the efficiency of various noise reduction features. Comparing our results with findings it appears that sensitivity to reconstructed NDVI to changing gaps in time series can be a subject of future studies if raw NDVI from satellite is used.

Fig. 8
figure 8

Sensitivity of SG Filter NDVI to the data interval

Full size image

The correlation in Table 2 does not show a clear trend when the interval is varied. This is primarily due to changing data resolution at different intervals. The most important to note is the R2 when the NDVIF and NDVISG at daily intervals are compared. A stronger correlation exists at daily intervals when the polynomial order was 4 (R2 = 0.82) than at the polynomial order of 3 (R2 = 0.78). This reinforces the earlier inference that the optimum polynomial order is 4 while applying SG Filter.

By taking the NDVIF series as a reference, Table 2 presents the correlation coefficient for various NDVI series. The correlation explains how correctly the trend of NDVI from the two sources agrees with each other. The correlation coefficients are also calculated by using piece-wise data set for 5 phenological series to highlight its tendency along the crop life cycle. The paired data points in the case of comparing NDVIS are 21 but the number of data points in phenological phase 1 and 3 are too small to calculate R2. The data points in SG Filter 1 day is sufficient to calculate R2 in all phenological phases.

The results in Table 2 show that overall a high correlation between NDVIF and NDVIS. We noted a much low R2 value for Field 3 (Fig. 4) when only paired data was considered. Processing all daily images from the field camera has improved the resolution of data points which has substantially enhanced agreement in NDVI trends (R2 = 0.91). The correlation coefficient in the emergence and maturity phases show high values of 0.87 and 0.99 but a slightly weak correlation is noted in the flowering phase (R2 = 0.65, n = 9). The correlation coefficient between NDVIF and NDVISG at 1 day interval is based on the largest number of data points (n = 131) and still shows high R2 values of 0.82 (polynomial order = 4) and 0.78 (polynomial order = 3). In Table 2, a closer look at SG Int01Ord4 shows that R2 is strong (> 0.8) for all phenological phases except the Sowing and Flowering phases. It is difficult to find the exact reasoning for these weak correlations.

The results presented in Table 2 are important as they suggest that NDVIS provides a correct trend of wheat crop health. The absolute values of NDVIS are much lower than the NDVIF nevertheless it can be used for crop identification and classification in areas of similar agro-climatic conditions as defined in various reports (e.g. Ahmad et al. (2019)). Satellite data is accessible and inexpensive compared to NDVIF acquired with the help of phenocams. The regression method can be applied with reasonable accuracy to apply correction to NDVIS. An OLS regression is applied to daily NDVIF and NDVISG (Interval 1, polynomial order = 4) as shown in Fig. 7. The resultant regression equation is used to plot a line that is a close depiction of NDVIF. Such types of corrections are useful in studies detecting signals of crop diseases, failure, or studies focussed on the estimation of yield/biomass.

Crop classification

Figure 9 shows classified satellite image. Approximately, 1850 km2 (24%) in Sargodha district comprises of wheat fields, while the Official (Agriculture Department, Punjab) statistics report a slightly higher area of 1732 km2 (29%) for the year 2022. In Table 3 consumer and producer accuracies for wheat are 97.6%. High accuracy is also found for remaining two classes. Overall effciency (97.5%) shows that high proportion of all reference sites was classified correctly in the settellite image. On the other hand, Kappa score is generated from a statistical test to evaluate the accuracy of a classification. Kappa score evaluate how well the classification performed as compared to just randomly assigning values. Kappa score value is 0.96 which is close to 1 denoting that classification using RFML algorithm in GEE did better than random.

Fig. 9
figure 9

Classification of wheat in Sargodha district using random forest machine learning algorithm in Google Earth Engine

Full size image
Table 3 Accuracy matrix of image classification using Random Forest Machine Learning Algorithm
Full size table

The crop area reported in official statistics is gathered through field surveys as well as by interviewing farmers, landowners, and agricultural communities. These sources provide valuable information about the types of crops planted, their locations, and the acreage dedicated to each crop. However, it is important to note that reported data can sometimes be prone to errors due to incomplete or inaccurate reporting, administrative delays, or inconsistencies in data collection methods. Satellite-based estimates are highly valuable as they offer almost real-time data, cover vast geographical regions, and remain consistent across areas. However, these estimates do have limitations, such as the challenge of accurately distinguishing between specific crops or the impact of cloud cover variations on image quality.

Conclusions

In this experimental study, daily NDVI-enabled field cameras monitored NDVI over four experimental fields over three crop seasons on a daily frequency. NDVI signatures of four crops were thus captured during phenological stages of the crops. Ideally one NDVI image is available from satellite sources but due to cloud cover, fog, and other factors it is rarely achievable for the entire crop period. Hence the resolution of satellite data is often not very high. The comparison of NDVI from field cameras and satellite images provides a useful insight into reliability and accuracy of NDVI from satellite sources. Generally, NDVI from satellite sources has been lower than that from field cameras. The strongest linear relationship between satellite and field NDVI has been found for wheat followed by maize, sugarcane, and rice respectively.

The gap in NDVI time series from satellite images negatively impacts reliable interpretation of crop health or yield estimation from NDVI during phenological stages. Therefore, various gap filling techniques and smoothening filters have been applied to NDVI time series of wheat crop. SG Filter has shown the best results. The sensitivity analysis for various parameters of SG filter indicates that the most sensitive parameter is polynomial order followed by the uniform interval used in the time series data. The sensitivity analysis is performed with reference to NDVI time series from field cameras. The absolute values of satellite NDVI are much lower than the NDVI from phonecams. Nevertheless, it can be used for crop identification and classification in areas of similar agro-climatic conditions as demonstrated in the analysis for Sargodha district. Agro-climatic conditions and suitability of crops in different agro-ecological zones is discussed in detail by Ahmad et al. (2019) which may be used to identify suitable areas for future studies. The accuracy (both consumer and producer) during classification has been high. The comparison of area under wheat crop from image classification with independent data set collected by a government agency also shows good agreement and provides confidence in the methods adopted in this paper.

Satellite data is accessible and inexpensive compared to using phonecams. However, NDVI time series from phonecams provide an insight into crop health during its phenological stages. The regression method can be applied with reasonable accuracy to apply correction to satellite NDVI which may be used to detect signals of crop diseases, failure, and to estimate of yield/biomass. Such analysis can be an interesting addition to this work for which future studies are encouraged.