Fuzzy machine learning approach for transitioned building footprints extraction using dual-sensor temporal data

This study presents a fuzzy approach, for detection of transitioned building footprints in urban area using medium resolution datasets. Multi-temporal remote sensing data sets from Landsat-8 Operational Land Imager and Sentinel-2A were used for generation of temporal indices database. The database was generated using class-based sensor independent-normalized difference vegetation index approach, with an aim to reduce spectral dimensionality of each image and maintain temporal dimensionality. The temporal indices database was subsequently used as input in Modified Possibilistic c-means classifier for transitioned building footprints extraction. The identified transitioned building locations were validated using ground samples as well as from Google images at four different test sites. For accuracy assessment, F-measure was calculated and its value was 0.75 or higher for all training and testing sites. Thus, using proposed fuzzy approach, transitioned building footprints were accurately identified compared to traditional techniques.


Introduction
India is witnessing a rapid and unplanned growth of its cities with urban population constituting 32% of total country population [1]. This unregulated pattern of growth is leading to loss of open spaces both in the city core and urban fringes, which in turn is disrupting various ecological and hydrological cycles. In order to ensure sustainable urban development, regular monitoring of open spaces and their transitions is important. The application of remote sensing data acquired by drones, aircrafts or satellites, for monitoring land cover transformation has become modern way of observing the world. In recent years, the availability of free remote sensing data (i.e., Landsat, Sentinel, MODIS etc.) has led to its wide application in urban land cover monitoring. Using data fusion techniques, data from different satellites can be assimilated for generation of temporal data sets especially, for land cover dynamics monitoring [2,3].
For information extraction, remote sensing images are classified into different classes using hard and soft classification techniques [4,5]. Hard classification techniques classify images into crisp classes and are applied for classifying homogenous pixels (i.e., pixels having only one class). Soft classification techniques generates the likelihood of each pixel belonging to various classes and are frequently applied in classification of mixed pixels (i.e., pixels containing more than one class) [6]. Fuzzy classifiers belong to the class of soft classifiers and are used in classification of objects whose definition is vague [7,8].
The pixel in urban areas are mixed in nature as they are composed of various land cover classes viz., concrete, bitumen, vegetated areas etc. The traditional hard classifiers been proven to be ineffective in handling this heterogeneity especially in coarse and medium resolution imageries [9,10]. On the other hand in fuzzy domain initially classifiers like, Fuzzy c-means (FCM) and Possibilistic c-means (PCM) were applied to handle mixed pixels [11]. FCM is an unsupervised clustering algorithm which has been widely used for fuzzy classification [12]. But it has a limitation, as it assigns the value relative to total number of classes [13]. The PCM addressed this shortcoming by assigning, the membership values to each pixel irrelevant to the number of classes present [14,15].
Various authors have reported studies for transitioned buildings footprints identification using post classification comparison of temporal classified images [16,17]. However, post classification change detection approach is depended on the accuracy of individual date classification outputs. The present study aims to address these shortcomings by using temporal indices (generated using remote sensing datasets) as input into the FCM classifier for extracting new building footprints [18,19].
The present paper is organized in nine (9) sections. Section 1 contains explanation about classification approaches and research objectives, Sect. 2 explains the details of the proposed fuzzy algorithm. Section 3 explains about role of indices used in the study. Section 4 illustrates the study area while Sect. 5 discusses the data sets used in this research work. Section 6 explains the proposed methodology, while Sect. 7 illustrates the study results. Sections 8 and 9 deal with discussion and conclusion of study results respectively.

Fuzzy based modified possibilistic c-means (MPCM)
Fuzzy c-Means (FCM) is one of the earliest clustering algorithms based on fuzzy set theory. It works on the assumption that number of clusters 'c' is known for the given dataset and attempts to minimize the objective function [20,21]. The main disadvantage of this algorithm is that it follows hyperline constraint [22,23]. FCM assigns each pixel a degree of sharing for each class. Thus, overall sum of the membership values becomes one. This is called as hyperline constraint, due to which it cannot handle noise present in the data [14,19]. In Possibilistic c-Means (PCM), the objective function of the FCM is modified and probabilistic constraint is also relaxed [14]. It is a clustering algorithm which can be applied as supervised mode while providing class centroid from training data [24]. The membership values generated through this objective function for each class are independent. However, the PCM requires good initialization [25] and generates coincident of clusters and has several advantages over FCM [13]. The memberships generated by PCM depend on degree of "belongingness" and the algorithm does not follow the hyperline constraint. However, the major drawbacks of PCM algorithm are [26,27]: • It is very sensitive to good initialization. • It has a property to generate coincident clusters because of rows and columns of typicality matrices are free from one another. • The algorithm minimizes the impact of noise but ignores the membership value that makes the class centroid close to the pixels or data points.
Modified Possibilistic c-Means (MPCM) [27] algorithm was suggested to overcome the shortfalls of FCM and PCM. The algorithm has a fast clustering ability and is more robust to noise and outliers. The algorithm is a modified form of PCM algorithm [19,27] and fits clusters which are close to each other. In present research work MPCM has been applied on temporal indices database.
For a given dataset X = X 1 , X 2 , X 3 … X N having clusters between 1 < c < N membership values of each clusters can be achieved by minimizing the objective function in Eq. (1) where 0 ≤ μ ik ≤ 1 , D ik = ∥ x k − v i ∥ , c is the number of clusters (or classes), N is the number of data points, μ ik is the typicality value of x k in class i and η i is the scale or distribution parameter which depends on the all data which is computed by the Eq. (2): where β is always chosen to be 1, and μ ik, FCM is the terminal membership value of FCM. In this case of MPCM the typicality values μ ik and cluster centers v i were obtained as stated in Eq. where all the variables in Eq. (1) to (4) are; J MPCM is objective function. X is image data set having X 1 , X 2 ,…..X N .
c is number of classes. N is total number of pixels in image. µ ik is membership value of i th class, k th pixel. D ik is square of Euclidean distance.
x k is unknown feature vector. v i is cluster center called mean feature vector. k is k th pixel. i is i th pixel (in this research work only one class). η i is the scale or distribution parameter of ith class. M weight constant, controls fuzziness in output. J MPCM is objective function in Eq. (1), which is minimized through partial differentiation. Partial differentiation of J MPCM objective function gives Eq. (2) and (3). Equation (2) generates η i called scale or distribution parameter of i th class. Scale or distribution parameter controls shape and size of cluster. Initially D ik is calculated between x k and v i vector elements. As mentioned x k is unknown feature vector representing vector elements as indices values from each temporal image (T1 and T2 images). While v i represents mean feature vector generated from training sample vectors. Elements of v i also represent vector elements as indices values from each temporal image (T1 and T2 images). Using D ik and η i , µ ik as membership of class has to be calculated using Eq. (3).
In present study, input data for MPCM algorithm x k and cluster center v i carry's two vector elements from the spectral information of the multi-temporal datasets, representing indices values from temporal images. The first vector element contains the information of open spaces from T1 dataset and second vector element contains the information of building footprint from T2 dataset at same pixel location. Using these two elements, algorithm generates, fraction images depicting membership values of building footprints which transitioned from open spaces.

Class-based sensor independent (CBSI)normalized difference vegetation index (NDVI)
In present study, a modified version of normalized difference vegetation index (NDVI) viz., Class-based Sensor Independent (CBSI-NDVI) was used (Eq. 6) [2], as it reduces the spectral dimensionality of multi-temporal datasets while preserving the temporal dimensionality of temporal indices database.
where max represents the band of maximum reflectance and min represents band of minimum reflectance. The value of CBSI ranges from 0 to + 1. The index was calculated, using the in-house JAVA based Sub-pixel Multispectral Image Classifier (SMIC) tool.

Study area
Dehradun city located in Uttarakhand state of India is the study area for the research work. The city has a geographical extend of 30° 15′ 18.362″ N to 30° 20′ 54.693″ N and 77° 58′ 1.464″ E to 78° 6′ 13.978″ E. Dehradun is the primate city in the region and the neighboring towns and settlements have a symbiotic relationship with Dehradun city depending on it, for higher level services. The city has thus emerged as a vital service centre, since the trade and commerce requirements of the region and higher order facilities of health, education, recreation and transportation of the surrounding hinterland are met by the city. As a result, during the last few decades, the city has registered an unprecedented sprawl in its area and the city is expanding rapidly onto the surrounding fertile agricultural lands. These development pressures have stressed the Dehradun city to the breaking point, besides disturbing various hydrological and ecological cycles and loss of arable land. Figure 1 shows the key-map for the study area. For training the algorithm fifty (50) samples were collected from site 1 (Bharuwala Colony as training site). As per Jensen [28], number of training samples per class should be 10n, where 'n' is dimensionality of data. In present study, since two dimensional indices database was taken as input to the MPCM classifier, the value of n = 2. However, we took the training samples to be 50 instead of 20, to have a better representation of transitioned building footprints.
Seventy five (75) testing samples were collected from four testing sites (Indian Institute of Remote sensing (IIRS) Kalidas Road, Doon University area, Sabji Mandi area and ISBT as testing sites). As per Congalton [29], 75-100 testing samples per class are sufficient, that's why 75 testing samples were collected from four different sites.

Data used
Images acquired by operational land Imager (OLI) sensor on board Landsat 8 and multi-spectral instrument (MSI) sensor on board Sentinel-2A (Table 1) were used in the study. The Landsat 8 sensor data was acquired on 20-10-2013 (T1) and Sentinel-2A on 22-10-2019 (T2). The time difference between the two datasets was taken to be 06 (six) years, to ensure that sufficient number of transitioned buildings footprints from open spaces are available.
The Landsat 8 multi-spectral data of T, was merged with panchromatic data of Landsat 8 satellite using wavelet  fusion algorithm [30,31]. The wavelet based approach generated fused data at 15 m which was subsequently resampled at 10 m, to ensure compatibility with Sentinel-2A data.

Methodology
The CBSI-NDVI index was calculated using the SMIC tool (refer Sect. 3). The assignment of spectral bands for calculating CBSI-NDVI for Landsat-8 and Sentinel-2A data has been shown in Table 2. These temporal CBSI-NDVI values were used for generating the temporal indices database, which was subsequently, used for training the MPCM classifier. For training the algorithm fifty (50) samples were collected from site 1 (Bharuwala Colony). Seventy five testing samples were collected from four testing sites, (Indian Institute of Remote sensing (IIRS) Kalidas Road, Doon University area, Sabji Mandi area and ISBT). Thus, the proposed approach incorporates multi-temporal information as a training signature in the Modified Possibilistic c-Means (MPCM) classifier. The methodology of this research work is depicted in Fig. 2. As can be observed from methodology chart, in comparison to the traditional post classification change detection algorithms, in proposed approach there is no need to classify images of time T1 and T2 separately.
For evaluation of the results, a maximum likelihood classification (MLC) was also performed on the images of time T1and T2 and NDVI values were also calculated for both the images. These classified images were then used for carrying out the post classification change detection.

Results
The results obtained using the MPCM classifier and the MLC classifier are shown in Fig. 3. It can be observed from Fig. 3d that MLC classifier is unable to separate the buildings from the paved surface areas and bare ground. While in sharp contrast using MPCM classifier the building footprints come out very sharply and smaller buildings are also detected (Fig. 3e). High resolution Google Earth images were used for validation purpose. The NDVI difference   (Fig. 3c), depicts the areas where vegetation change has taken place. It can be observed from Google Earth images that, areas where vegetation has increased, brighter shades are mapped and where vegetation transit to built-up, darker shades were mapped. These changes were nicely classified using the fuzzy MPCM classifier (red color). Four sites were considered for testing results of the proposed methodology (refer Sect. 4). Results for one of the test site having high built-up density (Indian Institute of Remote sensing (IIRS), on Kalidas Road) is shown in Fig. 4.
It can be observed from NDVI difference image (Fig. 4c), that NDVI values have decreased but the building footprints cannot be picked up. Newly transitioned small building footprint (red circle), which is IIRS Dining Hall, was extracted by the MPCM. Left bottom circled Shivalik Exotica apartment building (p) was also extracted which has transitioned from open ground. Building footprints (q, r) in fuzzy MPCM output were newly built IIRS Hostels. Footprint (s) is a stand-alone dwelling unit which has changed its roof surface material has also been detected. Footprint (t) is again apartment building named Meridian Heights, which transitioned from vegetated land. Figure 5 shows the classification results for another test area (Doon University) having low built-up density. Circles (p) and (q) shows the small and large transitioned building footprints, which are detected using MPCM. Very small newly transitioned buildings highlighted in the rectangle (r) were also identified and distinguished by fuzzy MPCM classification approach. Similar results were obtained for third test site (i.e., Sabji Mandi area) and fourth test site (ISBT) refer Figs. 6 and 7 respectively. Accuracy assessment measures used in the study are depicted in the Table 3, which are precision, recall, F-measure accuracy (Eq. 7) and kappa coefficient. Precision talks about the rightly classified values out of total predicted values and recall is explained as rightly classified values out of total actual values. F-measure [32] is the harmonic mean of the precision and recall and seeks the balance between the two. If both the precision and recall are high, value of F-measure increases and vice-versa. Similarly overall accuracy and kappa coefficient were also calculated. The F-measure has been defined in Eq. (7).  For f-measure to be evenly balanced, β should be 1. Both F-measure and accuracy are quite high at training site as well as at all the testing sites for fuzzy MPCM algorithm, whereas using post-classification change detection the accuracies were low for building footprint extraction. Accuracy assessment for NDVI change detection was excluded in this study as it only gave the values which show decrease or increase in the vegetation on the land.

Discussion
Studies by [33,34] have demonstrated the application of high resolution data in extraction of transitioned building footprint from open spaces. However, to the best of our knowledge no study has been reported, where transitioned building footprint were detected using freely available coarse spatial resolution imageries. Currently, post-classification change detection is widely used for the analysis of land cover change. The study by [35] explains the change detection analysis and its application in urban area. NDVI change detection method is based on the assumption that urban area and vegetation are negatively proportionate with each other for a particular pixel. Both of them require, user opinion in selecting the bands for the process which in turn introduces subjectivity. The present study compares the results of three techniques namely, NDVI based, post classification and MPCM classifier for detecting the transitioned building footprints extraction using multi-temporal data. The MPCM classifier has less subjectivity and also handles the mixed pixel problem.
The principle objective of this study was to extract transitioned buildings from open spaces in different parts of Dehradun city using soft classification approach, in single step. Added value from this research work is that fuzzy based MPCM algorithm can be trained with very small training database. The study was conducted using two dates multi-temporal datasets of Landsat 8 (year 2013) and Sentinel-2A (year 2019).
Various authors have attempted to study transitioned building footprint using conventional technique like, Maximum Likelihood Classifier (MLC) classifier, where images are classified as crisp classes. The results showed that conventional classification techniques mix the builtup/buildings with the bare ground. While assessing   in identifying the built-up which was transitioned from vegetation. But it failed to separate the built-up and bare ground. These issues were successfully addressed in proposed approach using MPCM classifier. Small transitioned building footprints could not be extracted as the first image (T1) is of coarser resolution (i.e., 30 m). Secondly, if the images (of T1 and T2) were from same sensor, it would result in improved image to image registration which would again result in better pixel to pixel correspondence between the two temporal images, eventually may lead to more accurate transitioned building footprints extraction. High resolution images can give better results in extracting small transitioned building footprints such as residential houses and shops. Some noisy output pixels can be removed using Markov Random Field (MRF) or local convolution methods which add contextual information in the base classifier.

Conclusion
In conventional change detection techniques multi-temporal satellite images are initially classified into respective land cover classes, hereafter these classified images were compared to detect the changes occurred. This is a twostep process i.e., classification followed with change detection. Using these conventional technique, built-up change can be mapped from medium spatial resolution remote sensing data however it is not possible to map transitioned building footprint. Secondly in case of coarse resolution images mixed pixels introduce classification errors, which are further propagated in change detection process.
In NDVI change detection approach also, it was possible to map vegetated area. But it is not feasible to map transitioned building footprints from open area. The present study proposes a fuzzy based approach for detection of transitioned building footprints from open space, while handling mixed pixels from coarse resolution data. The study concludes that mapping of the newly transitioned buildings footprints from open spaces had been possible using fuzzy machine learning algorithm and temporal indices data. In case of proposed fuzzy MPCM classification, the subjectivity is reduced, as it requires very few training samples and the algorithm selects the maximum and minimum reflectance bands for the target class during indices generation, to reduce spectral dimensionality of multi-temporal images. Hence, the present study demonstrates the applicability of proposed fuzzy machine learning algorithm in urban areas where open spaces are rapidly transiting to built-up areas. As future scope, detection of the transitioned building footprints of smaller buildings can be carried out using finer spatial resolution temporal dataset.
Authors' contribution All three authors contributed equally as team members.
Data availability All data was available freely from Landsat-8 and Sentinel -2A site.
Code availability Code for proposed methodology was written by Authors.

Compliance with ethical standards
Conflict of interest The authors declare that they have no conflict of interest.
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creat iveco mmons .org/licen ses/by/4.0/.