1 Introduction

Presently, researchers are searching for potential solutions to control COVID-19. Coronaviruses are mainly experts at reducing the immunity of persons. In the period of viral infection, the host person opposes the virus by obtaining immunity [1]. Modern medicine has very little to suggest in managing COVID-19 patients. But Ayurveda has a bundle of immunity-boosting techniques for ongoing pandemic situations [2]. The word Ayurveda originated from Ayur (life) and Veda (science or knowledge) two Sanskrit words [3]. Ayurveda is the traditional system of medicine in India. Ayurveda provides more importance on constructing the power of the mind.

Ayurveda helps to prevent different kind’s strains causing our body including diseases. In Ayurveda, immunity is defined as the capability of preventing the evolution of diseases. In Ayurveda, immunity is classified into mainly three types. They are namely natural, chronological, and acquired [4]. Incorrect selection and usage of plants directly affect the usage of the medicine. This may lead to side effects for the consumers. To avoid this situation, strict and accurate plant recognition should be followed.

The Botanists consider the flowers, leaves, root seeds, stems, etc. to identify the plants. Leaves are an easily available part of the plant, so leaves are the most used part for plant identification. The advanced machine learning technology reduces the burden of the complexity of leaf image-based plant recognition. Under the COVID-19 like situations, the mobile application for medicinal plants identification is seems to be most essential component.

Few recently developed algorithms like shape descriptor algorithm for medicinal plant identification (SDAMPI) [5] is highly supportable for mobile applications. This is a lossless algorithm which accurately describes the leaf shape information. This digital descriptor is based on freeman chain code [6] and conserves rotational transactional and resizable data of images and guarantees suitable information for the future. It is based on shape numbers which can provide invariance for rotational and scale-based changes. Numeric-based descriptors result in ease of handling and reduced database size compared to other existing descriptors. The SDAMPI [5] descriptor is strictly designed for closed shapes. In the shape descriptor generation algorithm, every neighbour pixel is extracted and denoted using 0 to 7 digits which represent the position of neighbour pixels in the shape boundary [5]. The shape descriptor generation for the leaf image is shown in Fig. 1.

Fig. 1
figure 1

The Shape Descriptor generation for a single leaf

Full size image

2 Common Ayurvedic Plants used for Immunity Boosting

Indian traditional ayurvedic remedies have the expertise of several centuries to develop immunity in the human being. The Ministry of AYUSH of Govt. of India suggested that the combination of Ocimum tenuiflorum, Cinnamomum verum, Piper nigrum, Zingiber officinale, and Vitis vinifera (Holy Basil, Cinnamon, Black Pepper, Ginger, and Grape Vine) as a protector against “COVID-19” [7]. In [8], authors have defined the immune system as the forerunner of life. The authors of [8] explained that the Ayurveda does not limit plants as healing procedures rather provides a lot of importance on the practice of therapies that will keep a person in a positive health state. The authors of [8] also listed a set of immune-boosting plants and their applications in detail. The process of identification and collection of plants from nature for immunity boosting and treatment is a beneficial component of the Indian culture which helps people to maintain good health. The experience of plant identification is a part of the exciting factor of the social tradition of many people.

A table of common Ayurvedic plants and parts used for immunity boosting is described in [9]. The selected plants for developing a mobile application for medicinal plant identification are shown in Fig. 2. The scientific details of the selected plants are given in Table 1.

Fig. 2
figure 2

Medicinal plants used in the application

Full size image
Table 1 List of medicinal plants used in this application
Full size table

2.1 Ocimum Sanctum

In Ayurveda, the Ocimum sanctum is called Tulsi or Holy basil. This is an antimicrobial and antiviral herb used for sputum balancing in our body and useful in cough and cold. Tulsi is a sub-shrub with green or purple leaves. Leaves are arranged in simple opposite with ovate in shape and toothed margins, up to 5 cm long [10].

2.2 Phyllanthus Amarus

Phyllanthus amarus is a well-known small herb of the Indian Ayurvedic system. It is mainly used in gastropathy use like stomach problems and genitourinary systems. In Ayurveda, Phyllanthus amarus is also used to treat ulcers and wounds [11].

2.3 Aloe Vera

Aloe vera is widely used in food and cosmetics. The Antioxidant effects of the inner gel of the Aloe vera leaves make a more significant role and are known as the healing plant of the Indian Ayurvedic system. Aloe vera is a stemless or very short-stemmed plant with thick and fleshy leaves with green to grey-green in colour. Leaves border is serrated and has very tiny white teeth [12].

2.4 Carica Papaya

The papaya is a small, lightly branched tree with spirally arranged leaves. The leaves are large, deeply palmately lobed, with seven lobes. Leaves extracted from Carica papaya, have antioxidant properties and are used as a tonic for the heart, analgesia and treatment for stomach aches. The unripe fruit of Carica papaya is used in diabetes treatment. Fruits of Carica papaya contain many antioxidant nutrients, and minerals and are a good source of fibre. Fruit is used for the treatment of arthritis, ringworm, hypertension etc. [13].

2.5 Centella Asiatica

Centella asiatica have kidney-shaped green leaves around 2 cm and stems are slender green to reddish green in colour. This plant is commonly used as spice, vegetables, and juice. In traditional medicine, it is used for wound therapy because it has anti-wrinkle and anti-inflammation properties. Researchers are going over Centella asiatica's memory-enhancing and brain-enlightening roles [14].

2.6 Baccopa Monnieri

In Indian Ayurvedic tradition, Baccopa monnieri also known as Brahmi is used for improving intelligence and memory. Brahmi is also given to persons with anxiety, poor memory, epilepsy, asthma, ulcers and tumors. Brahmi is a creeping plant found in wet and muddy zones, leaves are thick, oblong-shaped and are oppositely arranged on the stem [15, 16].

2.7 Azhadiracta Indica

Azhadiracta Indica is a fast-growing tree also known as Neem which is a rich source of antioxidants. Leaves of Neem are arranged opposite, with a pinnate shape. In Ayurveda Neem is used for skin diseases, to balance blood sugar levels, to improve liver function etc. In Indian tradition, neem branches are used as toothbrushes and tongue cleaners [17].

2.8 Zingiber Officinale

Zingiber officinale is also known as Ginger. In Indian culture Ginger is widely used as a spice. The warming effects, antibacterial, antifungal, anti-ulcer, and immunomodulatory properties are utilized in many Ayurvedic medicines. In many countries, Ginger is used as medicine for digestive problems. It is a reed plant-like, with a leafy stem [18].

2.9 Leucas Aspera

Leucas aspera also known as Thumba is an annual herb. Leaves are arranged Opposite with linear or narrowly oblong, shapes. Thuma has a fever-reducing capacity. This plant is crushed and widely used as inhaled in Ayurveda [19].

2.10 Pseudarthria Viscid

Pseudarthria viscid is an herb used for asthma, treatment of insect bites, vomiting etc. This plant is also known as Moovila in the Malayalam language because of its special three leaves arrangement. Leaves have a trifoliate, ovate-oblong shape [20].

3 Existing Mobile Applications for Plant Identification

Pl@ntNet [21] android application is available in Google play for identifying many cultivated plants. This application is designed by scientists from France. Pictures of android phones are chosen here for identification. Whenever one picture is compared with the database, the main database grows, and new images get reviewed by expert persons.iNaturalist [22], application, providing herbarium and related information for plant researchers. This application is helpful for field data collection and specimen identification. The huge database of this app provides high accuracy and improves the use of the collected data. This application allows users to record their observations as photos, and audio. This data information includes a combination of field pictures and a collection of metadata like the location of data collection, date and time which can easily be helpful for reviewers. This is a public method of taxonomic documentation and authentication that is driven by researchers from the California Academy of Sciences and the National Geographic Society.

PlantSnap [23] application acts as an interface between people and nature and provides complete information on given photos of plants which includes type, flowers, fruit and source etc. PlantSnap also collects information from the application users and that information is used for educational purposes. This deep learning engine image identification application provides highly accurate results for its users.

In [24], an IOS-based mobile application for automated recognition of plants and flowers based on convolutional neural networks is discussed. In this, after data collection, the data cleaning and data purification steps are applied before loading the sample into the deep learning algorithm. The performance of the application is tested over Cloud-based resources to increase the speed of image detection. A convolutional neural networks-based mobile application for beginners to identify plant species using leaves is developed by [25]. In this, the size of 1280 × 960 images is reduced to 320 × 240 images to further reduce the computational complexity. Three CNN models, VGG19, MobileNet, and MobileNetV2 are used to analyze the accuracy of the application. In [26], a shape descriptor-based mobile application using plant leaf is proposed. In this method, a multiscale shape descriptor is generated from leaves using contour, convexity, and concavity of leaf shape. The performance of mobile applications is tested over the Swedish leaf dataset and their dataset called Leaf100. Offline databases integrated into this mobile application allowed users to work continues without a network connection. SIFT features-based, scale and rotational invariant model and Bag of Word (BOW) model, for reducing the high dimensionality of the data space, using the Support Vector Machine (SVM) classifier for plant recognition are also being used [27]. In this android mobile application, shape-based digital morphological features of leaf images are analyzed to generate shape descriptors. In [28], a leaf-based plant identification system for android mobile phones using supervised learning is proposed. In this supervised classifier method, SURF features and Bag of Words features are used to analyze the leaf features.

From the above study of existing plant identification, artificial intelligence-based mobile applications are time-consuming as well as very complicated. Artificial intelligence-based mobile applications require a huge number of images and processing speed. However, in the medicinal plant identification field, artificial intelligence-based mobile applications should provide highly accurate results. So, the main objectives of this mobile application development are: to reduce the processing complexities of deep learning-based medicinal plant analysis and achieve high accuracy in medicinal plant identification.

4 Methodology

In this paper, leaf descriptors are analyzed based on a deep learning algorithm to identify the medicinal plant. Layers of neurons are trained to classify names that describe images into classes. Deep convolutional neural networks are suitable to categorize numeric data, without knowledge of descriptor structure.

4.1 Deep Learning

Many deep learning networks and the importance of convolutional neural networks are discussed in [29]. A hand position identification in images using the support of a faster R-CNN object detector is implemented in [30]. The importance of deep learning algorithms in the sensor-based environmental monitoring field is proposed in [31]. The effectiveness of deep learning methods is evaluated over fluid images and an extraction technique for velocity fields from images is discussed in [32]. In [33], a deep neural network-based fusion method is proposed by combining general features and adaptive features of images for preserving the robust performance of smart fault analysis. To learn the potential features of defects and obtain the location of defects with high accuracy, an efficient convolutional neural network is described in [34]. Due to low brightness and heavy obstruction circumstances in tree detection tasks, common object sensors cannot be straight useful for automatic road tree detection. An integrated start-to-finish teachable network based on an advanced deep learning-based tree object sensor is given in [35]. In [36], an innovative learning method for surface fault examination of images, both convolutional neural networks and the light characteristic-based learning method are combined and utilized. This sequence identification method does not contain any feature selection or sequencing procedures [37]. Therefore, this procedure improves the speed of image prediction as well as the accuracy of the result. In deep learning neural networks, multiple features are compared concurrently. For that, the image descriptor is divided into different lengths.

4.2 Database Generation

To recognize plants from their pictures, the internal database of this application is trained with leaf image shape descriptors of different Immunity boosting plants, which are collected from Kottiyoor village, Kannur District, Kerala State, India. Java programming language based on this android application will work on almost all android phones higher than version 4.0. Android Studio integrated development environment is used to build this mobile application. Single-leaf images are considered for the data collection. In this shape descriptor generation mechanism, each image is acquired using the Samsung A7 model android mobile phone and saved in 32 × 32 sized format for processing. To get highly accurate results and easy processing, each photo is captured in the daytime, by keeping a whiteboard in the leaf background and without plucking or harming the leaf. The details of dataset are as follows: Input data type: JPG, Size: 32 × 32, No. of Category: 10, Total No. of Samples: 1000, Data used for Training: 800, Data used for Validation: 300, Data used for Testing:200.

4.3 Proposed Method

A leaf descriptor generation is the first phase of the system. The selected descriptor is then analyzed using bigram based deep-learning algorithm for leaf identification. For the training and testing of the leaf image, a descriptor generated by SDAMPI, a deep-learning network with a convolution operation, max pooling, and a fully connected network is used. A bi-gram set of leaf shape numbers is convoluted using a base kernel and the result is passed to the max pooling stage to collect the maximum generated result value. The max pool result is sent into a fully connected network to decide the leaf category. The architecture of the deep learning network used here is a simple and basic convolutional network given in Fig. 3.

Fig. 3
figure 3

Architecture of SDAMPI based Deep Learning Network

Full size image

4.4 Descriptor Analysis using Deep Learning

Here, in this method, two adjacent digits also called bigrams are used. So, this can help to control the repeated digits. Each layer compares the descriptor’s each digit’s groups separately. In Deep learning-based descriptor identification, the whole descriptor is not inputted into the neural network, the input descriptor is partitioned into overlying tiles. In the content searching time, a fixed-sized descriptor or kernel will scan over and then applies a network to all bigram and calculated outcomes to describe which portion of the descriptor is most noteworthy. So, the maximum counted bigrams are selected in the conclusion. In Fig. 4, the architecture of bigram and deep learning-based prediction is given. The descriptor identification method using deep learning is given in Fig. 5.

Fig. 4
figure 4

Architecture of bigram and deep learning-based prediction

Full size image
Fig. 5
figure 5

Descriptor identification method using the deep learning

Full size image

4.5 Mobile Application

The complete process is implemented on the Android platform so that the user of the mobile application can identify the plants on- the site. Plants included in Table 1 are used in the training phase to detect medicinal plants. After installation, the application will ask for permission to use a mobile camera and storage for capturing the live image for identification. If the user allows permission for the camera, the dashboard activity of the application will open, here the user gets two options, one for plant identification and another for use of the digital library of the medicinal plants used in this application.

The application captures the leaf image and compares it with the digital library. If the application identifies the leaf in its library, provide the details of the plant to the user. Users can add images and information about new plants to the digital library provided in this application for enhancing the accuracy and identification rate of the mobile application. This android application can be extendable for user-defined image detection applications. The built-in database contains the details of plant leaves given in Table 1. But the user can add his necessary images into the database in the training phase, and those images are automatically converted into shape descriptors in the application itself. Hence, this mobile application can be used in other areas also. The screenshots of the mobile application are shown in Fig. 6.

Fig.6
figure 6

The screenshots of the mobile application

Full size image

5 Results and Discussions

The proposed mobile application and database for medicinal plant identification is focused to explain the importance of the shape descriptor mechanism in the image identification platform, where highly accurate results can be achieved from the image descriptor. The proposed android-based image detection system can also explain the importance of shape descriptors in the deep learning era. The internal database of this application is trained using 32 × 32 images. Increased sizes like 64 × 64 and 128 × 128 sized images have a lengthier shape descriptor than the trained shape descriptor so the result shows slightly varied accuracy in detection rate. For 64 × 64 sized images this mobile application achieved 96% accuracy and 87% accuracy is achieved for 128 × 128 sized images. In Fig. 7, the performance evaluation and result analysis of the proposed mobile application using our collected images database are given.

Fig. 7
figure 7

The performance evaluation of the proposed mobile application

Full size image

5.1 Evaluation using Other Databases

The leaf shape descriptor can highly support the image detection capability of artificial intelligence by providing an accurate explanation for the image features. This application is also tested over the “Flavia” and the “Swedish” data sets [38]. The proposed mobile application is also able to train with a user-defined database. For the performance evaluation, the proposed application is trained with popular databases namely Flavia[39] and Swedish[40]. The result analysis is performed over the plants as mentioned in Fig. 8.

Fig. 8
figure 8

Details of result analysis of this application

Full size image

5.2 SDAMPI Leaf Shape Descriptor vs. HOG, SIFT and SURF Features

In [41], the authors explained the Importance of Histograms of oriented gradients (HOGs) for detecting a person from digital images; many authors utilized the same HOG descriptor generation method for their gradient-based descriptor generation. In the HOG descriptors generation, the input image is first divided into small cells and a histogram of pixel values in each cell is calculated. Each histogram is normalized based on a block-wise model for descriptor generation. Finally, all descriptors are arranged to generate a complete HOG descriptor. This HOG descriptor generation is more time-consuming. Rotated image identification is a limitation of HOG descriptors. A lot of background information also will be added to the HOG descriptor.

SIFT [42] technique is based on neighbour pixel features detected through a step-by-step filtering approach. Mainly SIFT descriptors are used for image retrieval. The disadvantages of SIFT descriptors are that SIFT descriptor generation method and comparison methods are very time-consuming. HOG is used for human detection and SIFT is used for image retrieval.

In [43], authors used Speeded Up Robust Features (SURF) features and in [44], the authors used the Bag of Features which is related to SURF features. The SURF features have a high error rate. While compared with other descriptors, the SDAMPI leaf shape descriptor [5] is easy to implement and rotated images can also be easily identifiable. These descriptors are depending on the morphological information of the leaf image and are mainly used for morphological features-based leaf identification.

5.3 The Invariance of Shape Number

A detailed study of the invariance property of basic Shape numbers is described by H. Freeman in his article [6]. An example of leaf shape descriptors generated with the help of SDAMPI shape descriptors with leaf images with different geometric transactions is given in Table 2. After the scaling, resizing and rotational transactions, the same shape number is generated. After vertical flipping and horizontal flipping, results have shown slightly different shape numbers but that result is resolved after the convolution and max pooling process. Shape number preserves the shape of the object in the entire process. Shape number provides the greatest representation of images in deep learning.

Table 2 NDWT denoised images for facial regions
Full size table

5.4 Comparison with Existing Mobile Applications

This is a mobile application for plant detection. The proposed mobile application provides a highly accurate result with the help of deep learning and leaf shape descriptors. Most of the state of art methods used the image directly but in the proposed method, in the classification part, the deep learning networks used the lite numerical leaf descriptor as input so that the performance of the mobile application for medicinal plant identification gives fast results. The image identification part is easy and fast because the database of the application contains a numerical descriptor for the image details. Digital descriptors reduced the database size. Morphological feature-based comparison improved the detection accuracy. This mobile application provides plant information like the AI-based Kisan Se Kisan Tak (KSKT) Mobile Application as proposed in [45]. Figure 9. shows the comparison between different mobile applications’ detection accuracy.

Fig. 9
figure 9

Comparison between different mobile applications plant detection accuracy

Full size image

5.5 Advantages

The mobile application discussed in this paper provides highly accurate results in the field of medicinal plant identification using image processing. The detailed database of immunity-boosting medicinal plants and their benefits are also discussed. The proposed shape descriptor generation for the deep learning method is very lite as compared to the existing image descriptors, hence, the mobile application can provide more accurate plant identification in real-time. To solve the network connectivity issues and to access the database in hills, forests, and rural areas, the proposed method works offline. Users can install the proposed application on their mobile phones and use it in remote areas.

5.6 Limitations

The proposed mobile application can provide highly accurate results using the SDAMPI [5] based shape descriptor and open shape descriptor algorithm. However, the descriptor generation step is time-consuming according to the image size. Currently, a 32 × 32 sized image format is used for image processing because a pixel-by-pixel analysis of the image takes more time compared to the existing image descriptors. Normal deep learning uses the whole image but a separate generation of leaf image shape features for accurate detection requires more time and cost.

6 Conclusion

In this paper, we proposed a deep learning, shape descriptor and open shape descriptor-based mobile application for immunity-boosting medicinal plant identification. This mobile application showed highly accurate results to detect immunity-boosting medicinal plants. The proposed open shape descriptor method can further be generalized for the identification of rare species that will help botanists. In the proposed method, “Flavia” and “Swedish” databases are used for testing the mobile application.

The morphological shape features of leaf images are evaluated during training and testing which provides a highly accurate result for this method. To make the system more usable and user-friendly, a database training provision is provided. Also, to overcome the network connectivity issues in hill, forest or rural areas, the offline application process can be used for medicinal plant identification. The performance of the proposed mobile application achieved higher accuracy results than existing deep learning-based image identification methods.

In future work, more complex leaf features also will be combined into the leaf descriptor to enhance the overall performance of the plant identification. A future extension to this work would also be the enhancement of a database for more plant species.