Towards Improved Air Quality Monitoring Using Publicly Available Sky Images

  • Eleftherios Spyromitros-XioufisEmail author
  • Anastasia Moumtzidou
  • Symeon Papadopoulos
  • Stefanos Vrochidis
  • Yiannis Kompatsiaris
  • Aristeidis K. Georgoulias
  • Georgia Alexandri
  • Konstantinos Kourtidis
Part of the Multimedia Systems and Applications book series (MMSA)


Air pollution causes nearly half a million premature deaths each year in Europe. Despite air quality directives that demand compliance with air pollution value limits, many urban populations continue being exposed to air pollution levels that exceed by far the guidelines. Unfortunately, official air quality sensors are sparse, limiting the accuracy of the provided air quality information. In this chapter, we explore the possibility of extending the number of air quality measurements that are fed into existing air quality monitoring systems by exploiting techniques that estimate air quality based on sky-depicting images. We first describe a comprehensive data collection mechanism and the results of an empirical study on the availability of sky images in social image sharing platforms and on webcam sites. In addition, we present a methodology for automatically detecting and extracting the sky part of the images leveraging deep learning models for concept detection and localization. Finally, we present an air quality estimation model that operates on statistics computed from the pixel color values of the detected sky regions.



This work is partially funded by the European Commission under the contract number H2020-688363 hackAIR.


  1. 1.
    Aiello, L.M., Petkos, G., Martin, C., Corney, D., Papadopoulos, S., Skraba, R., Göker, A., Kompatsiaris, I., Jaimes, A.: Sensing trending topics in twitter. IEEE Transactions on Multimedia 15(6), 1268–1282 (2013)CrossRefGoogle Scholar
  2. 2.
    Alexandri, G., Georgoulias, A., Meleti, C., Balis, D., Kourtidis, K., Sanchez-Lorenzo, A., Trentmann, J., Zanis, P.: A high resolution satellite view of surface solar radiation over the climatically sensitive region of eastern mediterranean. Atmospheric Research 188, 107–121 (2017)CrossRefGoogle Scholar
  3. 3.
    Alexandri, G., Georgoulias, A., Zanis, P., Katragkou, E., Tsikerdekis, A., Kourtidis, K., Meleti, C.: On the ability of regcm4 regional climate model to simulate surface solar radiation patterns over europe: an assessment using satellite-based observations. Atmospheric Chemistry and Physics 15(22), 13,195–13,216 (2015)CrossRefGoogle Scholar
  4. 4.
    Bay, H., Ess, A., Tuytelaars, T., Gool, L.V.: Speeded-up robust features (surf). Computer Vision and Image Understanding 110(3), 346–359 (2008). Similarity Matching in Computer Vision and MultimediaCrossRefGoogle Scholar
  5. 5.
    Chatfield, K., Simonyan, K., Vedaldi, A., Zisserman, A.: Return of the devil in the details: Delving deep into convolutional nets. arXiv preprint arXiv:1405.3531 (2014)Google Scholar
  6. 6.
    Ciresan, D., Giusti, A., Gambardella, L.M., Schmidhuber, J.: Deep neural networks segment neuronal membranes in electron microscopy images. In: Advances in neural information processing systems, pp. 2843–2851 (2012)Google Scholar
  7. 7.
    Donahue, J., Jia, Y., Vinyals, O., Hoffman, J., Zhang, N., Tzeng, E., Darrell, T.: Decaf: A deep convolutional activation feature for generic visual recognition. In: International conference on machine learning, pp. 647–655 (2014)Google Scholar
  8. 8.
    Farabet, C., Couprie, C., Najman, L., LeCun, Y.: Learning hierarchical features for scene labeling. IEEE transactions on pattern analysis and machine intelligence 35(8), 1915–1929 (2013)CrossRefGoogle Scholar
  9. 9.
    Ganin, Y., Lempitsky, V.: Nˆ 4-fields: Neural network nearest neighbor fields for image transforms. In: Asian Conference on Computer Vision, pp. 536–551. Springer (2014)Google Scholar
  10. 10.
    Georgoulias, A.K., Alexandri, G., Kourtidis, K.A., Lelieveld, J., Zanis, P., Pöschl, U., Levy, R., Amiridis, V., Marinou, E., Tsikerdekis, A.: Spatiotemporal variability and contribution of different aerosol types to the aerosol optical depth over the eastern mediterranean. Atmospheric Chemistry and Physics 16(21), 13,853 (2016)Google Scholar
  11. 11.
    Girshick, R., Donahue, J., Darrell, T., Malik, J.: Rich feature hierarchies for accurate object detection and semantic segmentation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 580–587 (2014)Google Scholar
  12. 12.
    Gupta, S., Girshick, R., Arbeláez, P., Malik, J.: Learning rich features from rgb-d images for object detection and segmentation. In: European Conference on Computer Vision, pp. 345–360. Springer (2014)Google Scholar
  13. 13.
    Hariharan, B., Arbeláez, P., Girshick, R., Malik, J.: Simultaneous detection and segmentation. In: European Conference on Computer Vision, pp. 297–312. Springer (2014)Google Scholar
  14. 14.
    Haywood, J., Boucher, O.: Estimates of the direct and indirect radiative forcing due to tropospheric aerosols: A review. Reviews of geophysics 38(4), 513–543 (2000)CrossRefGoogle Scholar
  15. 15.
    Igoe, D., Parisi, A., Carter, B.: Characterization of a smartphone camera’s response to ultraviolet a radiation. Photochemistry and photobiology 89(1), 215–218 (2013)CrossRefGoogle Scholar
  16. 16.
    Irfanullah, K.H., Sattar, Q., Sadaqat-ur Rehman, A.A.: An efficient approach for sky detection. IJCSI International Journal of Computer Science Issues 10 (2013)Google Scholar
  17. 17.
    Jacobs, N., Roman, N., Pless, R.: Consistent temporal variations in many outdoor scenes. In: Computer Vision and Pattern Recognition, 2007. CVPR’07. IEEE Conference on, pp. 1–6. IEEE (2007)Google Scholar
  18. 18.
    Kinne, S., O’Donnel, D., Stier, P., Kloster, S., Zhang, K., Schmidt, H., Rast, S., Giorgetta, M., Eck, T.F., Stevens, B.: Mac-v1: A new global aerosol climatology for climate studies. Journal of Advances in Modeling Earth Systems 5(4), 704–740 (2013)CrossRefGoogle Scholar
  19. 19.
    Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. In: Advances in neural information processing systems, pp. 1097–1105 (2012)Google Scholar
  20. 20.
    Li, Y., Huang, J., Luo, J.: Using user generated online photos to estimate and monitor air pollution in major cities. In: Proceedings of the 7th International Conference on Internet Multimedia Computing and Service, p. 79. ACM (2015)Google Scholar
  21. 21.
    Liu, C., Tsow, F., Zou, Y., Tao, N.: Particle pollution estimation based on image analysis. PloS one 11(2), e0145,955 (2016)CrossRefGoogle Scholar
  22. 22.
    Long, J., Shelhamer, E., Darrell, T.: Fully convolutional networks for semantic segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3431–3440 (2015)Google Scholar
  23. 23.
    Lowe, D.G.: Distinctive image features from scale-invariant keypoints. International journal of computer vision 60(2), 91–110 (2004)CrossRefGoogle Scholar
  24. 24.
    Markatopoulou, F., Mezaris, V., Patras, I.: Cascade of classifiers based on binary, non-binary and deep convolutional network descriptors for video concept detection. In: Image Processing (ICIP), 2015 IEEE International Conference on, pp. 1786–1790. IEEE (2015)Google Scholar
  25. 25.
    Moumtzidou, A., Papadopoulos, S., Vrochidis, S., Kompatsiaris, I., Kourtidis, K., Hloupis, G., Stavrakas, I., Papachristopoulou, K., Keratidis, C.: Towards air quality estimation using collected multimodal environmental data. In: International Workshop on the Internet for Financial Collective Awareness and Intelligence, pp. 147–156. Springer (2016)CrossRefGoogle Scholar
  26. 26.
    Ning, F., Delhomme, D., LeCun, Y., Piano, F., Bottou, L., Barbano, P.E.: Toward automatic phenotyping of developing embryos from videos. IEEE Transactions on Image Processing 14(9), 1360–1371 (2005)CrossRefGoogle Scholar
  27. 27.
    Olmo, F.J., Cazorla, A., Alados-Arboledas, L., López-Álvarez, M.A., Hernández-Andrés, J., Romero, J.: Retrieval of the optical depth using an all-sky ccd camera. Applied optics 47(34), H182–H189 (2008)CrossRefGoogle Scholar
  28. 28.
    Pan, Z., Yu, H., Miao, C., Leung, C.: Crowdsensing air quality with camera-enabled mobile devices. In: AAAI, pp. 4728–4733 (2017)Google Scholar
  29. 29.
    Pinheiro, P., Collobert, R.: Recurrent convolutional neural networks for scene labeling. In: International Conference on Machine Learning, pp. 82–90 (2014)Google Scholar
  30. 30.
    Poduri, S., Nimkar, A., Sukhatme, G.S.: Visibility monitoring using mobile phones. Annual Report: Center for Embedded Networked Sensing pp. 125–127 (2010)Google Scholar
  31. 31.
    Ricchiazzi, P., Yang, S., Gautier, C., Sowle, D.: Sbdart: A research and teaching software tool for plane-parallel radiative transfer in the earth’s atmosphere. Bulletin of the American Meteorological Society 79(10), 2101–2114 (1998)CrossRefGoogle Scholar
  32. 32.
    Saito, M., Iwabuchi, H.: A new method of measuring aerosol optical properties from digital twilight photographs. Atmospheric Measurement Techniques 8(10), 4295–4311 (2015)CrossRefGoogle Scholar
  33. 33.
    van de Sande, K.E.A., Gevers, T., Snoek, C.G.M.: Evaluating color descriptors for object and scene recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence 32(9), 1582–1596 (2010)CrossRefGoogle Scholar
  34. 34.
    Sarigiannis, D.A., Karakitsios, S.P., Kermenidou, M.V.: Health impact and monetary cost of exposure to particulate matter emitted from biomass burning in large cities. Science of The Total Environment 524, 319–330 (2015)CrossRefGoogle Scholar
  35. 35.
    Seinfeld, J.H., Pandis, S.N.: Atmospheric chemistry and physics: from air pollution to climate change. John Wiley & Sons (2016)Google Scholar
  36. 36.
    Sikora, T.: The mpeg-7 visual standard for content description-an overview. IEEE Transactions on Circuits and Systems for Video Technology 11(6), 696–702 (2001). 10.1109/76.927422CrossRefGoogle Scholar
  37. 37.
    Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 (2014)Google Scholar
  38. 38.
    Snoek, C., Cappallo, S., Fontijne, D., Julian, D., Koelma, D.C., Mettes, P., van de Sande, K., Sarah, A., Stokman, H., Towal, R., et al.: Qualcomm research and university of amsterdam at trecvid 2015: Recognizing concepts, objects, and events in video. In: NIST TRECVID Workshop (2015)Google Scholar
  39. 39.
    Spyromitros-Xioufis, E., Papadopoulos, S., Kompatsiaris, I., Tsoumakas, G., Vlahavas, I.: A comprehensive study over vlad and product quantization in large-scale image retrieval. IEEE Transactions on Multimedia (2014)Google Scholar
  40. 40.
    Stamnes, K., Tsay, S.C., Wiscombe, W., Jayaweera, K.: Numerically stable algorithm for discrete-ordinate-method radiative transfer in multiple scattering and emitting layered media. Applied optics 27(12), 2502–2509 (1988)CrossRefGoogle Scholar
  41. 41.
    Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., Rabinovich, A.: Going deeper with convolutions. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 1–9 (2015)Google Scholar
  42. 42.
    Tompson, J.J., Jain, A., LeCun, Y., Bregler, C.: Joint training of a convolutional network and a graphical model for human pose estimation. In: Advances in neural information processing systems, pp. 1799–1807 (2014)Google Scholar
  43. 43.
    Xiao, J., Hays, J., Ehinger, K.A., Oliva, A., Torralba, A.: Sun database: Large-scale scene recognition from abbey to zoo. In: Computer vision and pattern recognition (CVPR), 2010 IEEE conference on, pp. 3485–3492. IEEE (2010)Google Scholar
  44. 44.
    Zerefos, C., Gerogiannis, V., Balis, D., Zerefos, S., Kazantzidis, A.: Atmospheric effects of volcanic eruptions as seen by famous artists and depicted in their paintings. Atmospheric Chemistry and Physics 7(15), 4027–4042 (2007)CrossRefGoogle Scholar
  45. 45.
    Zerefos, C., Tetsis, P., Kazantzidis, A., Amiridis, V., Zerefos, S., Luterbacher, J., Eleftheratos, K., Gerasopoulos, E., Kazadzis, S., Papayannis, A.: Further evidence of important environmental information content in red-to-green ratios as depicted in paintings by great masters. Atmospheric Chemistry and Physics 14(6), 2987–3015 (2014)CrossRefGoogle Scholar
  46. 46.
    Zhang, C., Yan, J., Li, C., Rui, X., Liu, L., Bie, R.: On estimating air pollution from photos using convolutional neural network. In: Proceedings of the 2016 ACM on Multimedia Conference, pp. 297–301. ACM (2016)Google Scholar
  47. 47.
    Zhijie, Z., Qian, W., Huadong, S., Xuesong, J., Qin, T., Xiaoying, S.: A novel sky region detection algorithm based on border points. International Journal of Signal Processing, Image Processing and Pattern Recognition 8(3), 281–290 (2015)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG, part of Springer Nature 2018

Authors and Affiliations

  • Eleftherios Spyromitros-Xioufis
    • 1
    Email author
  • Anastasia Moumtzidou
    • 1
  • Symeon Papadopoulos
    • 1
  • Stefanos Vrochidis
    • 1
  • Yiannis Kompatsiaris
    • 1
  • Aristeidis K. Georgoulias
    • 2
  • Georgia Alexandri
    • 2
  • Konstantinos Kourtidis
    • 2
  1. 1.Centre for Research & Technology Hellas – Information Technologies InstituteThessalonikiGreece
  2. 2.Democritus University of ThraceXanthiGreece

Personalised recommendations