Skip to main content
SpringerLink
Log in

Automatic classification of rural building characteristics using deep learning methods on oblique photography

  • Research Article
  • Published:
Building Simulation Aims and scope Submit manuscript

Abstract

Rural building is important to the well-being of rural residents, leading to a significant need to carry out extensive surveys and retrofits of many rural buildings. On-site surveys by expert surveyors are currently the main approach, but this traditional method is often expensive and laborious, especially for large-scale survey tasks. Therefore, this study explores an alternative workflow based on deep learning (DL) methods to apply automatic classification of rural building characteristics. Taking four villages in Jizhou District of Tianjin, China as research samples, we tested selected convolutional neural network (CNN) architectures through the establishment of the training database containing 3258 labeled images, under the performance metrics of accuracy, recall and F1 score. The results showed that ResNet50 is the CNN architecture with the best performance, with the comprehensive consideration of overall metrics. Taking accuracy as the performance metric to test the generalization ability of ResNet50, the prediction results for seven building characteristic indicators from low to high are as follows: building function (0.827); building style (0.863); building quality (0.871); building age (0.880); building structure (0.891); abandoned or not (0.959); the number of storeys (0.995). Due to simplicity, accuracy and effectiveness, this workflow is transferable and cost-effective to investigate large-scale villages.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Bengio Y (2009). Learning Deep Architectures for AI. Boston: Now Publishers Inc.

    Book  MATH  Google Scholar 

  • Biljecki F, Ito K (2021). Street view imagery in urban analytics and GIS: A review. Landscape and Urban Planning, 215: 104217.

    Article  Google Scholar 

  • Cai J, Li B, Yu W, et al. (2020). Household dampness and their associations with building characteristics and lifestyles: Repeated cross-sectional surveys in 2010 and 2019 in Chongqing, China. Building and Environment, 183: 107172.

    Article  Google Scholar 

  • Chen L-C, Barron JT, Papandreou G, et a. (2016). Semantic image segmentation with task-specific edge detection using CNNs and a discriminatively trained domain transform. In:Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA.

  • Cheng G, Han J (2016). A survey on object detection in optical remote sensing images. ISPRS Journal of Photogrammetry and Remote Sensing, 117: 11–28.

    Article  Google Scholar 

  • Cheng G, Yang C, Yao X, et al. (2018). When deep learning meets metric learning: remote sensing image scene classification via learning discriminative CNNs. IEEE Transactions on Geoscience and Remote Sensing, 56: 2811–2821.

    Article  Google Scholar 

  • Chollet F (2017). Xception: Deep learning with depthwise separable convolutions. In:Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA.

  • Dai M, Ward WOC, Meyers G, et al. (2021). Residential building facade segmentation in the urban environment. Building and Environment, 199: 107921.

    Article  Google Scholar 

  • Diakogiannis FI, Waldner F, Caccetta P, et al. (2020). ResUNet-a: A deep learning framework for semantic segmentation of remotely sensed data. ISPRS Journal of Photogrammetry and Remote Sensing, 162: 94–114.

    Article  Google Scholar 

  • Dian R, Li S, Guo A, et al. (2018). Deep hyperspectral image sharpening. IEEE Transactions on Neural Networks and Learning Systems, 29: 5345–5355.

    Article  MathSciNet  Google Scholar 

  • Dong C, Loy CC, He K, et al. (2014). Learning a deep convolutional network for image super-resolution. In: Proceedings of European Conference on Computer Vision.

  • Dong B, Wang X (2016). Comparison deep learning method to traditional methods using for network intrusion detection. In: Proceedings of 2016 8th IEEE International Conference on Communication Software and Networks (ICCSN), Beijing, China.

  • Fan C, Yan D, Xiao F, et al. (2021a). Advanced data analytics for enhancing building performances: From data-driven to big data-driven approaches. Building Simulation, 14: 3–24.

    Article  Google Scholar 

  • Fan Y, Ding X, Wu J, et al. (2021b). High spatial-resolution classification of urban surfaces using a deep learning method. Building and Environment, 200: 107949.

    Article  Google Scholar 

  • Fang X (2020). On the achievements of building a well-off society in an all-round way from the perspective of residents’ income and expenditure. People’s Daily, 2020-07-27, P. 10. (in Chinese)

  • Gawrys MR, Carswell AT (2020). Exploring the cost burden of rural rental housing. Journal of Rural Studies, 80: 372–379.

    Article  Google Scholar 

  • Gkartzios M, Scott M, Gallent N (2020). Rural housing. In: Kobayashi A (Ed.), International Encyclopedia of Human Geography, 2nd edn. Amsterdam: the Netherlands.

  • Gong F-Y, Zeng Z-C, Zhang F, et al. (2018). Mapping sky, tree, and building view factors of street canyons in a high-density urban environment. Building and Environment, 134: 155–167.

    Article  Google Scholar 

  • Gonzalez D, Rueda-Plata D, Acevedo AB, et al. (2020). Automatic detection of building typology using deep learning methods on street level images. Building and Environment, 177: 106805.

    Article  Google Scholar 

  • Guo R, Liu J, Li N, et al. (2018). Pixel-wise classification method for high resolution remote sensing imagery using deep neural networks. ISPRS International Journal of Geo-Information, 7(3): 110.

    Article  Google Scholar 

  • Haapio A, Viitaniemi P (2008). A critical review of building environmental assessment tools. Environmental Impact Assessment Review, 28: 469–482.

    Article  Google Scholar 

  • He K, Zhang X, Ren S, et al. (2016). Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.

  • Himeur Y, Ghanem K, Alsalemi A, et al. (2021). Artificial intelligence based anomaly detection of energy consumption in buildings: A review, current trends and new perspectives. Applied Energy, 287: 116601.

    Article  Google Scholar 

  • Höhle J (2021). Automated mapping of buildings through classification of DSM-based ortho-images and cartographic enhancement. International Journal of Applied Earth Observation and Geoinformation, 95: 102237.

    Article  Google Scholar 

  • Hu C-B, Zhang F, Gong F-Y, et al. (2020). Classification and mapping of urban canyon geometry using Google Street View images and deep multitask learning. Building and Environment, 167: 106424.

    Article  Google Scholar 

  • Hu Q, Zhen L, Mao Y, et al. (2021). Automated building extraction using satellite remote sensing imagery. Automation in Construction, 123: 103509.

    Article  Google Scholar 

  • Huang J, Zhang X, Xin Q, et al. (2019). Automatic building extraction from high-resolution aerial images and LiDAR data using gated residual refinement network. ISPRS Journal of Photogrammetry and Remote Sensing, 151: 91–105.

    Article  Google Scholar 

  • Jin X, Davis CH (2005). Automated building extraction from high-resolution satellite imagery in urban areas using structural, contextual, and spectral information. EURASIP Journal on Applied Signal Processing, 2005: 2196–2206.

    MATH  Google Scholar 

  • Jin Y, Yan D, Chong A, et al. (2021). Building occupancy forecasting: A systematical and critical review. Energy and Buildings, 251: 111345.

    Article  Google Scholar 

  • Johnston CJ, Andersen RK, Toftum J, et al. (2020). Effect of formaldehyde on ventilation rate and energy demand in Danish homes: Development of emission models and building performance simulation. Building Simulation, 13: 197–212.

    Article  Google Scholar 

  • Kamath CN, Bukhari SS, Dengel A (2018). Comparative study between traditional machine learning and deep learning approaches for text classification. In: Proceedings of the ACM Symposium on Document Engineering, Halifax NS, Canada.

  • Kang J, Körner M, Wang Y, et al. (2018). Building instance classification using street view images. ISPRS Journal of Photogrammetry and Remote Sensing, 145: 44–59.

    Article  Google Scholar 

  • Kang X, Yan D, An J, et al. (2021). Typical weekly occupancy profiles in non-residential buildings based on mobile positioning data. Energy and Buildings, 250: 111264.

    Article  Google Scholar 

  • Kong L, Liu Z, Wu J (2020). A systematic review of big data-based urban sustainability research: State-of-the-science and future directions. Journal of Cleaner Production, 273: 123142.

    Article  Google Scholar 

  • Leaman A, Stevenson F, Bordass B (2010). Building evaluation: practice and principles. Building Research & Information, 38: 564–577.

    Article  Google Scholar 

  • LeCun Y, Bengio Y, Hinton G (2015). Deep learning. Nature, 521(7553): 436–444.

    Article  Google Scholar 

  • Li E, Femiani J, Xu S, et al. (2015). Robust rooftop extraction from visible band images using higher order CRF. IEEE Transactions on Geoscience and Remote Sensing, 53: 4483–4495.

    Article  Google Scholar 

  • Li Y, Huang X, Liu H (2017). Unsupervised deep feature learning for urban village detection from high-resolution remote sensing images. Photogrammetric Engineering & Remote Sensing, 83: 567–579.

    Article  Google Scholar 

  • Lin T-Y, Goyal P, Girshick R, et al. (2017). Focal loss for dense object detection. In: Proceedings of 2017 IEEE International Conference on Computer Vision (ICCV), Venice, Italy.

  • Liu Y (2018). Introduction to land use and rural sustainability in China. Land Use Policy, 74: 1–4.

    Article  Google Scholar 

  • Liu Y, Chen X, Wang Z, et al. (2018a). Deep learning for pixel-level image fusion: Recent advances and future prospects. Information Fusion, 42: 158–173.

    Article  Google Scholar 

  • Liu Y, Fan B, Wang L, et al. (2018b). Semantic labeling in very high resolution images via a self-cascaded convolutional neural network. ISPRS Journal of Photogrammetry and Remote Sensing, 145: 78–95.

    Article  Google Scholar 

  • Liu J, Li T, Xie P, et al. (2020). Urban big data fusion based on deep learning: An overview. Information Fusion, 53: 123–133.

    Article  Google Scholar 

  • Lu Z, Im J, Rhee J, et al. (2014). Building type classification using spatial and landscape attributes derived from LiDAR remote sensing data. Landscape and Urban Planning, 130: 134–148.

    Article  Google Scholar 

  • Lu S (2021). Regional Architectural language recognition and evaluation based on visual perception: the case of minority housing in Nujiang area. New Architecture, 2021(01): 110–115. (in Chinese)

    Google Scholar 

  • Lu X, Feng F, Pang Z, et al. (2021a). Extracting typical occupancy schedules from social media (TOSSM) and its integration with building energy modeling. Building Simulation, 14: 25–41.

    Article  Google Scholar 

  • Lu Z, Wang T, Guo J, et al. (2021b). Data-driven floor plan understanding in rural residential buildings via deep recognition. Information Sciences, 567: 58–74.

    Article  Google Scholar 

  • Lyu P, Yu M, Hu Y (2020). Contradictions in and improvements to urban and rural residents’ housing rights in China’s urbanization process. Habitat International, 97: 102101.

    Article  Google Scholar 

  • Ma L, Liu Y, Zhang X, et al. (2019). Deep learning in remote sensing applications: A meta-analysis and review. ISPRS Journal of Photogrammetry and Remote Sensing, 152: 166–177.

    Article  Google Scholar 

  • Mangalathu S, Burton HV (2019). Deep learning-based classification of earthquake-impacted buildings using textual damage descriptions. International Journal of Disaster Risk Reduction, 36: 101111.

    Article  Google Scholar 

  • Marcos D, Volpi M, Kellenberger B, et al. (2018). Land cover mapping at very high resolution with rotation equivariant CNNs: Towards small yet accurate models. ISPRS Journal of Photogrammetry and Remote Sensing, 145: 96–107.

    Article  Google Scholar 

  • Na H, Choi H, Kim T (2020). Metabolic rate estimation method using image deep learning. Building Simulation, 13: 1077–1093.

    Article  Google Scholar 

  • Na R, Shen Z (2021). Assessing cooling energy reduction potentials by retrofitting traditional cavity walls into passively ventilated cavity walls. Building Simulation, 14: 1295–1309.

    Article  Google Scholar 

  • National Bureau of Statistics (2020). China Statistical Yearbook. Beijing: Chinese Statistics Press. (in Chinese).

    Google Scholar 

  • Porikli F, Shan S, Snoek C, et al. (2018). Deep learning for visual understanding: part 2. IEEE Signal Processing Magazine, 35(1): 17–19.

    Article  Google Scholar 

  • Preiser WF, Nasar JL (2008). Assessing building performance: Its evolution from post-occupancy evaluation. International Journal of Architectural Research, 2(1): 84–99.

    Google Scholar 

  • Rueda-Plata D, González D, Acevedo AB, et al. (2021). Use of deep learning models in street-level images to classify one-story unreinforced masonry buildings based on roof diaphragms. Building and Environment, 189: 107517.

    Article  Google Scholar 

  • Sun Y, Wang X, Tang X (2014). Deep Learning Face Representation from Predicting 10, 000 Classes. In: Proceedings of 2014 IEEE Conference on Computer Vision and Pattern Recognition, Columbus, OH, USA.

  • Tan M, Le Q (2019). EfficientNet: Rethinking model scaling for convolutional neural networks. In: Proceedings of International Conference on Machine Learning.

  • Tian W, Zhu C, Sun Y, et al. (2021). Energy characteristics of urban buildings: Assessment by machine learning. Building Simulation, 14: 179–193.

    Article  Google Scholar 

  • Venetianer PL, Werblin F, Roska T, et al. (1995). Analogic CNN algorithms for some image compression and restoration tasks. IEEE Transactions on Circuits and Systems I: Fundamental Theory and Applications, 42: 278–284.

    Article  Google Scholar 

  • Wang C, Yan D, Jiang Y (2011). A novel approach for building occupancy simulation. Building Simulation, 4: 149–167.

    Article  Google Scholar 

  • Wang X, Zhao Y, Pourpanah F (2020). Recent advances in deep learning. International Journal of Machine Learning and Cybernetics, 11: 747–750.

    Article  Google Scholar 

  • Watts AC, Ambrosia VG, Hinkley EA (2012). Unmanned aircraft systems in remote sensing and scientific research: classification and considerations of use. Remote Sensing, 4: 1671–1692.

    Article  Google Scholar 

  • Xie J (2019). Research on key technologies of rural building information extraction based on high resolution remote sensing images. PhD Thesis, Southwest Jiaotong University, China. (in Chinese)

    Google Scholar 

  • Xu X, Li J, Huang X, et al. (2016). Multiple morphological component analysis based decomposition for remote sensing image classification. IEEE Transactions on Geoscience and Remote Sensing, 54: 3083–3102.

    Article  Google Scholar 

  • Yan X, Ai T, Yang M, et al. (2019). A graph convolutional neural network for classification of building patterns using spatial vector data. ISPRS Journal of Photogrammetry and Remote Sensing, 150: 259–273.

    Article  Google Scholar 

  • Yin L, Wang Z (2016). Measuring visual enclosure for street walkability: Using machine learning algorithms and Google Street View imagery. Applied Geography, 76: 147–153.

    Article  Google Scholar 

  • Yong C (2020). Research on method and application of urban built-up area information extraction based on spectral index. Master Thesis, Chongqing University of Posts and Telecommunications, China. (in Chinese)

    Google Scholar 

  • Yu D, Ji S, Liu J, et al. (2021). Automatic 3D building reconstruction from multi-view aerial images with deep learning. ISPRS Journal of Photogrammetry and Remote Sensing, 171: 155–170.

    Article  Google Scholar 

  • Yuan Q, Wei Y, Meng X, et al. (2018). A multiscale and multidepth convolutional neural network for remote sensing imagery pan-sharpening. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 11: 978–989.

    Article  Google Scholar 

  • Yuan L, Guo J, Wang Q (2020). Automatic classification of common building materials from 3D terrestrial laser scan data. Automation in Construction, 110: 103017.

    Article  Google Scholar 

  • Zampieri A, Charpiat G, Girard N, et al. (2018). Multimodal image alignment through a multiscale chain of neural networks with application to remote sensing. In: Proceedings of the 15th European Conference Computer Vision, Munich, Germany.

  • Zhan S, Chong A, Lasternas B (2021). Automated recognition and mapping of building management system (BMS) data points for building energy modeling (BEM). Building Simulation, 14: 43–52.

    Article  Google Scholar 

  • Zhang X, Gao P, Zhao K, et al. (2020). Image restoration via deep memory-based latent attention network. IEEE Access, 8: 104728–104739.

    Article  Google Scholar 

  • Zhang F, Fan Z, Kang Y, et al. (2021a). “Perception bias”: Deciphering a mismatch between urban crime and perception of safety. Landscape and Urban Planning, 207: 104003.

    Article  Google Scholar 

  • Zhang X, Gao P, Liu S, et al. (2021b). Accurate and efficient image super-resolution via global-local adjusting dense network. IEEE Transactions on Multimedia, 23: 1924–1937.

    Article  Google Scholar 

  • Zhong B, Xing X, Love P, et al. (2019). Convolutional neural network: Deep learning-based classification of building quality problems. Advanced Engineering Informatics, 40: 46–57.

    Article  Google Scholar 

  • Zhou X, Tian S, An J, et al. (2021). Comparison of different machine learning algorithms for predicting air-conditioning operating behavior in open-plan offices. Energy and Buildings, 251: 111347.

    Article  Google Scholar 

Download references

Acknowledgements

This work was supported by the National Key R&D Program of China (2018YFD1100303).

Author information

Authors and Affiliations

  1. School of Architecture, Tsinghua University, Beijing, 100084, China

    Chengyu Meng, Jiaqi Ji, Ziyu Jia & Zhengxu Zhou

  2. School of Landscape Architecture, Beijing Forestry University, Beijing, 100083, China

    Chengyu Meng & Yuwei Song

  3. China National Engineering Research Center for Human Settlement, China Architecture Design & Research Group, Beijing, China

    Ziyu Jia

  4. College of Engineering, Peking University, Beijing, 100871, China

    Peng Gao

  5. School of Aerospace Engineering, Tsinghua University, Beijing, 100084, China

    Sunxiangyu Liu

Authors
  1. Chengyu Meng
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yuwei Song
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jiaqi Ji
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Ziyu Jia
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Zhengxu Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Peng Gao
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Sunxiangyu Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Zhengxu Zhou.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Meng, C., Song, Y., Ji, J. et al. Automatic classification of rural building characteristics using deep learning methods on oblique photography. Build. Simul. 15, 1161–1174 (2022). https://doi.org/10.1007/s12273-021-0872-x

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12273-021-0872-x

Keywords

Search

Navigation