Introduction

Global biodiversity has dramatically declined in recent decades, which has been described as the sixth major extinction event (Cowie et al. 2022). Invasive alien species (IAS) are among the main causes of biodiversity loss and extinction on both global and local levels (Roy et al. 2023). Once IAS are established, it is difficult or even impossible to eradicate them (Hulme 2006; Everts et al. 2022). However, IAS eradication is still feasible during the first stages of invasions when species occur at a low density (Simberloff 2003; Roy et al. 2009). Therefore, the prevention and early detection of IAS, before they become established, are the most effective methods in preventing invasions and reducing future problems (Leung et al. 2002; Everts et al. 2023). An inherent difficulty in monitoring IAS during the first stages of invasions is managing to effectively cover a large sampling area with limited funding (Ribeiro et al. 2022). This difficulty will increase in the future, since the number of IAS is expected to increase in the next few decades due to increasing trade, global transport, and climate change (Seebens et al. 2021). Hence, there is an urgent need for rapid, reliable, and effective early warning systems for IAS detection.

Recent development of automated and non-invasive tools in biomonitoring has greatly expanded opportunities for monitoring multiple IAS and covering a large spatial and temporal scale. These tools include the use of drones coupled with machine learning (Aota et al. 2021), satellite images analyses (Rajah et al. 2019), environmental DNA analyses (e.g. Sanz et al. 2023), and the combination of multiple non-invasive techniques (see Kamoroff et al. 2020). Most of these tools allow managers and conservationists to detect multiple species using a single method, which can reduce the project costs and increase the probability of detecting IAS (Ribeiro et al. 2022). Another relatively novel technique used in biomonitoring is passive acoustic monitoring (PAM), which uses autonomous sound recorders (Sugai et al. 2019; Desjonquères et al. 2020) to sample the soundscape either continuously or over a specific time period.

In fact, the use of PAM for biodiversity monitoring increased exponentially in recent years (Sugai et al. 2019). However, there are still only a few assessments of its effectiveness in detecting IAS (but see Rountree & Juanes 2017; Ribeiro et al. 2022; Amorim et al. 2023). One reason that may have hampered the use of PAM for IAS monitoring is that interpreting recordings can be very time consuming if the recordings need to be verified acoustically or visually (but see Cameron et al. 2020). However, in recent years, multiple automated software packages have appeared that use algorithms to identify candidate sounds. Some of the state-of-the-art techniques for automated recording analyses, such as deep learning and convolutional neural networks (Stowell 2022), might be difficult for managers and conservationists to implement without a computing or engineering background. However, there are a few user-friendly and ready-to-use machine learning approaches available (Manzano-Rubio et al. 2022; Ribeiro et al. 2022; Bota et al. 2023) that may increase the accessibility of these tools for managers and scientists working on IAS monitoring and management (Chen et al. 2023). Although the use of automated and species-specific acoustic recognizers may speed up the species detection, comprehensive assessments of the use of PAM for IAS monitoring are lacking (see Merlet et al. 2022).

In this paper, we aim to evaluate the utility and effectiveness of passive acoustic monitoring for detecting invasive alien species, using low-cost sound devices and a free and ready-to-use machine learning approach, BirdNET, which can provide an automated identification of over 6000 wildlife species (Kahl et al. 2021). To do this, we use the American bullfrog (Lithobates catesbeianus) as a model species, since it is vocally active, difficult to detect visually, and is considered to be one of the worst 100 IAS worldwide (Lowe et al. 2000). More specifically, we aimed to: (i) estimate the detection rate of American bullfrog presences (i.e. at least one call per recording), automatically detected by the recognizer, compared to manual analyses, (ii) assess the recognizer precision, defined as the percentage of BirdNET predictions correctly classified, and (iii) evaluate the amount of time needed for automated analyses and posterior validation of the software output. Additionally, we applied PAM, coupled with automated recognition software, to two real-world monitoring programmes to (i) describe the vocal behavior and identify the optimal time of the day for monitoring the American bullfrog in Europe (by monitoring one established population in Belgium and one in Italy), and (ii) evaluate the feasibility of using PAM and automated software for detecting the species presence on a large acoustic dataset collected within an official monitoring programme of the species on the Ebre Delta (Spain).We expect that our assessment may encourage managers and researchers to assess the effectiveness of PAM and automated software as an attractive monitoring method for monitoring IAS, and more specifically the American bullfrog.

Material and methods

Study species

The American bullfrog is native to eastern North America, but it has been introduced and successfully established in western United States of America, Canada, South America, Asia, and Europe (Ficetola et al. 2007). In Europe, free-ranging populations are already present in Belgium, France, Germany, Greece, and Italy (Ficetola et al. 2007). However, other climatically suitable areas not yet colonized by bullfrogs exist in Europe (Johovic et al. 2020), and therefore early-warning rapid response protocols are relevant for detecting this species. The American bullfrog has been linked to negative direct and indirect impacts on native species and communities through predation, competition, or spreading parasites and pathogens (e.g. Liu et al. 2015; Yap et al. 2018; Hossack et al. 2023). American bullfrog females may lay around 20,000 eggs per breeding attempt (Bury & Whelan 1985), and therefore once established bullfrog populations are often either difficult or impossible to eradicate (but see Kamoroff et al. 2020). For all these reasons, at the European level, the species is on the list of Invasive Alien Species of Union Concern (European Commission 2016).

The American bullfrog call is a loud sound of between three and fifteen sequential vocalizations, uttered at low frequencies (0.2–1.4 kHz, Supplementary Figure S1, Capranica 1968). During the breeding period, males usually call in isolation or in a large chorus, mainly during the hours after sunset and before sunrise (Laufer et al. 2017). Therefore detecting male vocal activity during the breeding season through acoustic surveys is a common method for monitoring the species (Laufer et al. 2017; Sanz et al. 2023). The species’ high vocal activity, together with its loud and characteristic call, suggest that PAM coupled with automated recognition software, would be a suitable and effective technique for detecting the species. However, there is no prior research on automated recognition in this species.

Case study A: Vocal activity pattern of the American bullfrog in Europe

During 2022 we recorded in two European wetlands with known free-ranging, breeding populations of the American bullfrog, one located in Belgium (Griesbroek, 51°08′ 41.3″ N 5°07′52.3″ E) and one in Italy (Villastrada, 44°58′ 5.86″ N 10°38′25.44″ E). At each wetland, we placed one Song Meter Micro recorder (250 EUR, Wildlife Acoustics, USA) attached to a 1 m wooden stick on the border of the wetland. In Belgium, the recorder was programmed to record nightly from sunset to sunrise during 11 consecutive days (from the 8th August until the 19th August, 217 30-min recordings), while in Italy the recorder worked nightly from sunset to sunrise during 12 consecutive days (from the 19th August until the 31st August, 250 30-min recordings). The recorders were programmed to record (in mono and wav format) in segments of 30 min during the whole daily sampling period, using a sampling rate of 22 kHz, gain of 18 dB, and 16 bits per sample (see recording protocol in Table 1). That case study was aimed to record during the whole night to identify the hours with the species’ maximum vocal activity, and therefore to identify the optimal period for monitoring the species; and to collect recordings in areas with known presence for further evaluation of the recognizer.

Table 1 Recording protocol followed in each country showing the recorder employed, survey period, total number of monitoring days and recording schedule. Recording schedule specifies the starting and ending recording time for each country, and the recording length. Recorders located in Belgium and Italy operated continuously, while in Spain the recorders were active from sunset to two hours after sunset and from two hours before sunrise to sunrise. It also shows the total number of recordings collected in each country
Full size table

Case study B: Long-term passive acoustic monitoring

During 2021, we placed one AudioMoth v. 1.2.0 recorder (100 EUR, Hill et al. 2018) in the north part of the Ebre Delta Natural Park (Catalonia, Spain). In this area, in June 2018, the first reported breeding event of the American bullfrog occurred in Spain, followed by a successful eradication programme. However, due to the species’ invasive character and the importance of its early detection, in 2018 the Natural Park initiated a monitoring programme coupling eDNA surveys with other field-based techniques (e.g., nocturnal surveys, active traps; see Sanz et al. 2023). In addition, in 2021 it initiated an acoustic monitoring programme aiming to increase the probability of detecting the species. The species’ density during 2021 was expected to be very low. The recorder operated from 28th June to 10th August (955 10-min recordings). It was placed inside an AudioMoth IPX7 case (Open Acoustic Devices) and attached to a 1 m wooden stick located on the border of the monitored wetland. The recorder was programmed to record (in mono and wav format), in segments of 10 min, during the two hours after sunset and during the two hours before sunrise (Laufer et al. 2017), using a sampling rate of 32 kHz, gain Med-High, and 16 bits per sample (see recording protocol in Table 1). The resulting acoustic dataset was used as a case study to evaluate the potential of passive acoustics and automated detection for monitoring potential sites, including an assessment of the time required for scanning a large acoustic dataset and validating its output (e.g. remove false positives).

Automated recording analyses

Acoustic recordings were analyzed using BirdNET (v2.2.0, Kahl et al. 2021), an automated sound classifier that is freely available on GitHub (https://github.com/kahst/BirdNET-Analyzer). BirdNET was originally developed for automated bird song recognition, but currently there are over 40 species-specific recognizers available for anurans on BirdNET, including the American bullfrog (see Wood et al. 2023; Pérez-Granados et al. 2023). BirdNET facilitates the automated detection and classification of wildlife vocalizations, through a deep neural network, using sound recordings (Kahl et al. 2021). BirdNET divides the recordings into 3-second segments and can identify over 6000 species of wildlife (Kahl et al. 2021), together with a quantitative confidence score ranging from 0 to 1. The confidence score is a user-selected parameter that can be used as a threshold value to filter the BirdNET output at a desired confidence level (see impact of confidence score on BirdNET output in Pérez-Granados 2023a). BirdNET also allows the user to adjust the overlap of prediction segments, the sensitivity parameter, and to apply filters to classify sounds only for a certain location, period, or target species (Kahl et al. 2021; Pérez-Granados 2023a).

We set BirdNET to classify sounds only for the American bullfrog, and thus avoid classifying sounds of non-target species (Manzano-Rubio et al. 2022). We ran BirdNET using the default values for the sensitivity parameter (1.0), overlap of prediction segments (no overlap, 0), and confidence score (threshold of 0.1). BirdNET was used to scan both the validation data set (see next section) and the data set collected in both case studies. The number of CPU threads used was set to 8 (default value of 4) with the aim of accelerating the acoustic analyses.

Recognizer performance

To assess the performance of BirdNET to detect the American bullfrog, we created a validation dataset of referenced recordings. To do this, we manually reviewed: (i) 50 30-min recordings from Belgium and Italy, resulting in a total of 100 30-min recordings, and (ii) the whole acoustic dataset (955 10-min recordings from) from Spain, for a total of 1055 recordings (12,550 min) validated across all sites. For each recording, an experienced observer annotated whether at least one call of the species was detected by a human using spectrograms in Raven Pro 1.6 (Cornell Lab of Ornithology 2023). For each recording with BirdNET predictions, we verified whether the species was really present. To do this, an observer verified a minimum of one BirdNET prediction per recording, by listening or inspecting at the timestamp of the 3 s spectrogram marked by BirdNET, and noted whether the species was present or absent at the selected timestamp. If the species was absent, a second detection was verified, and so on until the presence of the species was verified, or until the last detection was checked (see similar approach in Pérez-Granados et al. 2023). This procedure allowed us to estimate the percentage of presences detected by BirdNET, using default values, compared to manual analyses.

We also estimated BirdNET precision, a common acoustic metric used to assess recognizer performance (Knight et al. 2017). BirdNET precision was estimated by the rate of correctly classified BirdNET predictions to the total verified BirdNET predictions (Knight et al. 2017). To do this, we randomly selected and verified 614 BirdNET predictions from its output, including a minimum of one detection per recording (i.e., the one used to validate the species presence). We also verified a significantly larger number of predictions with low confidence scores (403 detections with a score < 0.4 and 211 detections with a score > 0.4). We followed this procedure due to the higher likelihood of false positives with lower confidence scores than those with a higher value (see Bota et al. 2023). For each prediction, the observer listened and inspected the timestamp of the 3 s spectrogram, and noted whether the American bullfrog was present or absent.

Finally, we also calculated the amount of time needed for scanning the whole acoustic dataset collected in Spain (the case study with the largest acoustic dataset), and the time needed to manually check the BirdNET output, and compared it to the amount of time needed for manual analyses.

Statistical analyses

To identify the hours with maximum vocal activity by the American bullfrog in Europe, we fitted a generalized additive model (GAM). GAM was fitted using the gam function of the “mgcv” package (Wood and Wood 2015) in R (v 3.6.2. R Development Core Team 2019) and the Gaussian family (selected after visually checking the raw data and model residuals, Supplementary Figure S2). This model was used in order to be able to model non-linear patterns as daily patterns of vocal activity. We used the number of BirdNET predictions per recording as the response variable, while recording hour and country were included as predictors. To more accurately represent the nature of the data, we modelled the predictor recording hour using an independent spline smoothed function for each monitored wetland.

Results

Recognizer performance

The presence of American bullfrogs was detected by a human in 90 of the 100 30-min recordings extracted from Italy and Belgium, and in five of the 955 10-min recordings made in Spain. BirdNET predicted the species’ presence in 86 of the 1055 sound recordings in the validation dataset, while no frogs were predicted in the Spanish dataset (Table 2). BirdNET only mislabelled the species presence in one recording (0.1% of the recordings with no presence annotated), which only had one detection and a low confidence score (0.116). Therefore, in 85 of the 86 (98.8%) recordings where the American bullfrog’s presence was predicted by BirdNET, the occurrence of the species was confirmed (Table 2). Overall, BirdNET detected the species’ presence in 85 of the 95 recordings annotated by a human. Consequently, the percentage of American bullfrog presences automatically detected by the recognizer, compared to a human, was 89.5% (see detailed results in Table 2).

Table 2 Confusion matrix of the ability of BirdNET to detect the presence of the American bullfrog in sound recordings. The confusion matrix is shown separately for each case study (Italy-Belgium, 100 30-min recordings manually reviewed; Spain, 955 10-min recordings manually reviewed), and for all recordings pooled (1055 recordings)
Full size table

BirdNET precision was 99.7%, since the species was verified in 612 of the 614 predictions checked. The two false positives had confidence scores of 0.116 and 0.156. Due to BirdNET’s high precision, hereinafter we decided to consider all BirdNET predictions as American bullfrog vocalizations.

Vocal activity pattern of the American bullfrog in Europe

The recorder placed in Belgium recorded 39,434 American bullfrog vocalizations in 11 days of monitoring, while the recorder in Italy recorded 15,691 vocalizations over 12 days. The American bullfrog’s vocal activity varied significantly between countries and recording hours (Table 3), with a significantly higher number of vocalizations detected in Belgium (Table 3 and see Supplemental Figure S3). The pattern of the species’ vocal activity was similar in both countries and was concentrated during the night, with lower activity during the crepuscular periods (Fig. 1). The species showed lower vocal activity around sunset, a continual increase in vocal effort as the night progressed, and a peak of vocal activity between two and three hours before sunrise. Vocal activity decreased afterwards (Fig. 1). In total, 4,960 vocalizations of the 55,125 vocalizations detected, occurred during the first two hours after sunset (8.9% of the total), with 10,083 (18.3% of the total) vocalizations registered during the two hours before sunrise, showing a clear pattern of vocalising during the central hours of the night (72.8% of the total).

Table 3 Summary table of a GAM model fitted to test the effect of recording hour and country on the vocal activity of the American bullfrog in Belgium and Italy
Full size table
Fig. 1
figure 1

Effect of recording time, expressed as hours recorded after sunset, on the vocal activity of the American bullfrog in A Belgium and B Italy. Estimates of the GAM model are plotted in different colors for each country. Vocal activity was monitored using passive acoustic monitoring during 11 and 12 consecutive nights, recording from sunset to sunrise, in Belgium and Italy, respectively, during August 2022. The colored areas represent associated 95% confidence intervals

Full size image

Long-term passive acoustic monitoring

The acoustic data set collected in Spain was of 159 h and 10 min. Visual scanning, and entering this data set took us around 59 working hours (37% of the amount of time recorded), while scanning time and BirdNET output verification took around 6 h (3.8% of the time recorded, a tenth of the time required for manual analyses). The recorded vocalizations of the American bullfrog on the five recordings with marked presence in Spain had low sound level, and therefore were most likely uttered by a single individual vocalizing far away from the recorder.

Discussion

In this study, we have demonstrated the effectiveness of integrating passive acoustic monitoring with a readily available and cost-free recognition algorithm, BirdNET. This combination proves to be a precise and efficient approach for automating the detection of the American bullfrog. The technique described here provides a valuable assessment of the effectiveness of a novel, and non-invasive technique for IAS detection. Indeed, the species is currently absent in many climatically suited areas and its potential distribution is expected to increase owing to global warming, especially in North America and central Europe (Johovic et al. 2020). Therefore, testing the effectiveness of automated techniques might be particularly well-suited for monitoring the American bullfrog in areas of potential expansion. Moreover, BirdNET is able to simultaneously predict thousands of wildlife species, and therefore it might be applied to monitor several IAS at once (Ribeiro et al. 2022). Our study is only one of a few assessing the effectiveness of BirdNET for detecting an anuran (but see Wood et al. 2023 and Pérez-Granados et al. 2023), so we hope it opens the new door for further studies using BirdNET (Pérez-Granados 2023a).

Using the default parameters, BirdNET detected around 90% of the occurrences of the American bullfrog in sound recordings and only annotated one mislabelled recording, which had a single prediction, and therefore the time needed for validating the BirdNET output of the only mislabelled recording was limited to a few seconds. We are aware that the long recording length of our recordings may have partly biased our results, but the high probability of detecting the presence of the species and the large number of vocalizations predicted in Belgium and Italy, supports the effectiveness of PAM and BirdNET in monitoring the American Bullfrog. Most of the BirdNET failures in detecting the bullfrog were likely related to the fact that the vocalizing individual was far away from the recorder, since the undetected signals had a low sound level. BirdNET’s ability to correctly identify bird vocalizations sharply decreases for vocalizations recorded farther than 50 m from the recorder (Pérez-Granados 2023b), and the same reasoning may apply to anurans. However, further research is needed on the distance at which the American bullfrog can be detected, as this may help managers and scientists to set up an adequate number of recorders when aiming to detect the species (see e.g. Perez-Granados et al. 2018). The signal-to-noise ratio (SNR) of the recorders is perhaps the greatest factor determining detection range of an ARU (Darras et al. 2020), which is also involved in the recording quality output. SNR for the AudioMoth is 44 dB (used in Spain), while SNR of the Song Meter Micro (used in Belgium and Italy) is about 73 dB. Therefore, the lower performance of our approach in Spain may also be related to the lower SNR of the ARU used. Since recording quality appears to be a crucial factor for automatically detecting calls, it would be advisable to use higher-quality recorders in areas with low population densities or potential species expansion, in order to maximize the probability of detecting these species, especially during the early stage of the invasions when the number of calls might be reduced.

On a posterior assessment we found that by increasing the overlap setting from 0 (default value) to 1 and by decreasing the confidence score threshold from 0.1 (default value) to 0.01, BirdNET detected the American bullfrog in 6 of the 10 undetected recordings from the validation dataset (including two from Spain). Therefore, BirdNET was able to detect the species’ presence in 91 of the 95 annotated recordings. Selecting the appropriate confidence score is a key point, since it has a significant impact on the output of BirdNET (see Bota et al. 2023). A higher confidence score can reduce false positives but also results in fewer predictions of the species’s occurrences (reviewed by Pérez-Granados 2023a). Further research may assess the trade-off between the percentage of occurrences detected and false positives annotated, in order to improve future surveys. However, according to our results, and to facilitate future surveys for managers and the public, we suggest running BirdNET using the default values because of the larger number of presences detected and the high precision acquired. Although some percentage of detections may be undetected using the default values, the species’ high vocal activity makes it difficult for it to remain undetected on larger spatial and temporal scales (e.g. at daily basis).

In the first case study, we used PAM to identify the hours of peak vocal activity of the American bullfrog in Europe. We are aware that our study only focused on a couple of locations and was performed at the end of the species’ breeding time in Europe. However, we hope that our findings might provide a useful baseline to guide the timing of future protocols and reduce the efforts (e.g. by reducing recording and scanning time and storage capacities) of future monitoring programmes. It might also serve as an example of how PAM can be applied to gain new insights into the ecology of IAS. The pattern of vocal activity of the American bullfrog was similar in Belgium and Italy, with higher vocal activity during the night, especially around 2–3 h before sunrise, and much lower activity during the crepuscular periods. The vocal pattern described here is similar to the one described for an invasive population of the American bullfrog in Uruguay (Laufer et al. 2017), suggesting that the higher vocal activity of the American bullfrog at night, in comparison to the crepuscular periods, might be a common behaviour trait in the species. Future studies may investigate this issue. Our findings suggest that the recording protocol followed in Spain (i.e. recording around sunset and sunrise) needs to be modified to extend the recording sessions into the night when the probability of detecting the species is maximised. The nocturnal vocal behavior of the American bullfrog, coupled with its dependence on aquatic habitats, makes this invasive species a suitable candidate species for using PAM in future monitoring programmes. This is because of the challenges associated with performing field surveys during night in wetlands (e.g. fragile vegetation, greater visual obstruction, boggy substrate; Manzano-Rubio et al. 2022).

In the second case study, we employed PAM and BirdNET to assess the effectiveness of this method to scan a large acoustic dataset collected as part of an official detection and eradication monitoring programme aimed at the species. Our results highlight the ability and speed of BirdNET to scan the acoustic dataset, due to high percentage of occurrences detected and reduced false positive rate. This suggests that this might be a suitable method for automated detection of the American bullfrog. According to our results, we also recommend users to set BirdNET to classify sounds only for the species of interest (6733 predictions made when BirdNET set to classify sounds only for the American bullfrog versus 67,999 predictions when considering the whole list of species in BirdNET). Further research is needed on identifying the optimal settings (including variable values of the sensitivity and overlap parameters) for detecting the American bullfrog. We set BirdNET to classify sounds only for the American bullfrog, and thus avoid classifying sounds of non-target species (Manzano-Rubio et al. 2022).

It is worth highlighting that it is possible to run BirdNET on a single-board computer Raspberry Pi (BirdNET-Pi, https://github.com/mcguirepr89/BirdNET-Pi), which will allow for automated recognition in real-time and to access data remotely through the web interface. BirdNET-Pi may allow managers and scientists to perform real-time species monitoring (i.e. early warning system) and to verify the presence of a species on a daily basis so that action can be taken almost immediately after detection. However, future studies may provide a more thorough evaluation of the pros and cons (e.g. elevated costs and more technical knowledge) of BirdNET-Pi when compared to running BirdNET on a computer.

PAM has been highlighted as a potential tool for detecting invasive species, but few examples have been developed (but see Ribeiro et al. 2022; Amorim et al. 2023). We provide a valuable assessment for remote, automated, and passive surveillance of the American bullfrog, which may have implications for monitoring and conservation programmes. In the current scenario of increasing biological invasions and an expected increase in the distribution of the American bullfrog (Johovic et al. 2020), we encourage managers and researchers to use PAM as a complementary, automated tool to improve the probability of the species’ detection.