Advertisement

Classification of Stereo Images from Mobile Mapping Data Using Conditional Random Fields

  • Max CoenenEmail author
  • Franz Rottensteiner
  • Christian Heipke
Original Article
  • 308 Downloads

Abstract

We propose a new method for the context-based classification of point clouds from stereo images using Conditional Random Fields (CRF). The classification is based on segments as nodes for the CRF. The segmentation is conducted on the image and is transferred to the 3D point cloud obtained by image matching. This allows the computation of 3D features additionally to the image features as well as the definition of realistic adjacencies between the segments in object space. We also propose a variant of the contrast-sensitive Potts model that is tailored for the contextual classification of point cloud segments. The evaluation of our method is performed on stereo sequences of a benchmark dataset, recorded in an urban area, and yields results with an overall accuracy of more than 90%. Moreover, we can show that the consideration of contextual information during the classification leads to an improvement of the overall accuracy.

Keywords

Stereo images Classification Segmentation 3D reconstruction Conditional random fields Mobile mapping 

Zusammenfassung

Klassifikation von Stereobildern aus Mobile Mapping Daten mittels Conditional Random Fields. In dieser Arbeit wird ein neues Verfahren zur kontextbasierten Klassifikation von Punktwolken aus Stereobildern mittels Conditional Random Fields (CRF) vorgestellt. Die Klassifikation setzt auf Segmenten als Knoten für das CRF auf. Die Segmentierung erfolgt im Bildraum und wird mittels einer 3D-Rekonstruktion der Szene auf die 3D-Punktwolke übertragen, was die Extraktion von 3D-Merkmalen zusätzlich zu den Bildmerkmalen sowie die Definition von realistischen Nachbarschaftsbeziehungen zwischen den Segmenten im Objektraum ermöglicht. Außerdem wird eine Variante des kontrastsensitiven Potts-Modells vorgestellt, welches für die kontextbasierte Klassifikation von Punktwolkensegmenten maßgeschneidert ist. Die Evaluierung der Methode erfolgt anhand von im urbanen Raum aufgenommenen Stereosequenzen eines Benchmark Datensatzes und liefert Ergebnisse mit einer Gesamtgenauigkeit von über 90%. Außerdem wird gezeigt, dass die Berücksichtigung von Kontext in der Klassifikation zu einer Erhöhung der Gesamtgenauigkeiten führt.

References

  1. Achanta R, Shaji A, Smith K, Lucchi A, Fua P, Snsstrunk S (2012) SLIC superpixels compared to state-of-the-art superpixel methods. IEEE Trans Pattern Anal Mach Intell 34(11):2274–2282CrossRefGoogle Scholar
  2. Aijazi AK, Checchin P, Trassoudaine L (2013) Segmentation based classification of 3d urban point clouds: a super-voxel based approach with evaluation. Remote Sens 5(4):1624–1650Google Scholar
  3. Anguelov D, Taskarf B, Chatalbashev V, Koller D, Gupta D, Heitz G, Ng A (2005) Discriminative learning of Markov random fields for segmentation of 3d scan data. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), vol 2, pp 169–176Google Scholar
  4. Bishop CM (2006) Pattern recognition and machine learning, 1st edn. Springer. New YorkGoogle Scholar
  5. Breiman L (2001) Random forests. Mach Learn 45(1):5–32CrossRefGoogle Scholar
  6. Brostow GJ, Shotton J, Fauqueur J, Cipolla R (2008) Segmentation and recognition using structure from motion point clouds. In: Proceedings of the european conference on computer vision (ECCV), vol. I, pp 44–57Google Scholar
  7. Chehata N, Guo L, Mallet C (2009) Airborne lidar feature selection for urban classification using random forests. In: International archives of the photogrammetry, remote sensing and spatial information sciences, vol XXXVIII-3/W8, pp 207–212Google Scholar
  8. Dohan D, Matejek B, Funkhouser T (2015) Learning hierarchical semantic segmentations of LIDAR data. In: International conference on 3D vision, pp 273–281Google Scholar
  9. Frey B, MacKay D (1998) A revolution: belief propagation in graphs with cycles. Adv Neur Inf Process Syst 10:479–485Google Scholar
  10. Geiger A, Lenz P, Urtasun R (2012) Are we ready for autonomous driving? The KITTI vision benchmark suite. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), pp 3354–3361Google Scholar
  11. Geiger A, Roser M, Urtasun R (2011) Efficient large-scale stereo matching. In: Proceedings of the 10th Asian conference on computer vision, vol I, pp 25–38Google Scholar
  12. Geman S, Geman D (1984) Stochastic relaxation, Gibbs distributions, and the Bayesian restoration of images. IEEE Trans Pattern Anal Mach Intell 6(6):721–741CrossRefGoogle Scholar
  13. Hackel T, Wegner JD, Schindler K (2016) Fast semantic segmentation of 3d point clouds with strongly varying density. In: ISPRS annals of the photogrammetry, remote sensing and spatial information sciences, vol III-3, pp 177–184Google Scholar
  14. Haralick RM, Shanmugam K, Dinstein I (1973) Textural features for image classification. IEEE Trans Syst Man Cybern SMC-3(6):610–621Google Scholar
  15. Heipke C, Mayer H, Wiedemann C, Jamet O (1997) Evaluation of automatic road extraction. In: International archives of photogrammetry and remote sensing, vol XXXII-3, pp 151–160Google Scholar
  16. Kang Y, Yamaguchi K, Naito T, Ninomiya Y (2009) Road scene labeling using SfM module and 3D bag of textons. In: 12th international conference on computer vision workshops (ICCV workshops), pp 657–664Google Scholar
  17. Kumar S, Hebert M (2006) Discrimantive random fields. Int J Comput Vis 68(2):179–201CrossRefGoogle Scholar
  18. Li X, Sahbi H (2011) Superpixel-based object class segmentation using conditional random fields. In: International conference on acoustics, speech and signal processing (ICASSP), pp 1101–1104Google Scholar
  19. Lim EH, Suter D (2009) 3D terrestrial LIDAR classifications with super-voxels and multi-scale conditional random fields. Comput-Aided Des 41(10):701–710CrossRefGoogle Scholar
  20. Lucchi A, Li Y, Smith K, Fua P (2012) Structured image segmentation using kernelized features. In: Proceedings of the European conference on computer vision (ECCV), vol II, pp 400–413Google Scholar
  21. Matti EK, Nebiker S (2014) Geometry and colour based classification of urban point cloud scenes using a supervised self-organizing map. Photogrammetrie Fernerkundung Geoinformation 3:161–173Google Scholar
  22. Munoz D, Bagnell JA, Hebert M (2012) Co-inference for multi-modal scene analysis. In: Proceedings of the European conference on computer vision (ECCV), vol VI, pp 668–681Google Scholar
  23. Munoz D, Vandapel N, Hebert M (2009) Onboard contextual classification of 3-D point clouds with learned high-order Markov random fields. In: IEEE international conference on robotics and automation ICRA ’09, pp 2009–2016Google Scholar
  24. Najafi M, Namin S, Salzmann M, Petersson L (2014) Non-associative higher-order markov networks for point cloud classification. In: Proceedings of the european conference on computer vision (ECCV), pp 500–515Google Scholar
  25. Niemeyer J, Rottensteiner F, Soergel U (2012) Conditional random fields for lidar point cloud classification in complex urban areas. In: ISPRS annals of the photogrammetry, remote sensing and spatial information sciences, vol I-3, pp 263–268Google Scholar
  26. Niemeyer J, Rottensteiner F, Sörgel U (2014) Contextual classification of Lidar data and building object detection in urban areas. ISPRS J Photogramm Remote Sens 87(2014):152–165CrossRefGoogle Scholar
  27. Reich M, Unger J, Rottensteiner F, Heipke C (2013) On-line compatible orientation of a micro-UAV based on image triplets. In: ISPRS annals of the photogrammetry, remote sensing and spatial information sciences, vol II-3/W2, pp 37–42Google Scholar
  28. Rusu RB, Holzbach A, Blodow N, Beetz M (2009) Fast geometric point labeling using conditional random fields. In: International conference on intelligent robots and systems, pp 7–12Google Scholar
  29. Sengupta S, Greverson E, Shahrokni A, Torr P (2013) Urban 3D semantic modelling using stereo vision. In: IEEE international conference on robotics and automation, pp 580–585Google Scholar
  30. Shapovalov R, Velizhev E, Barinova O (2010) Non-associative Markov networks for 3d point cloud classification. In: International archives of the photogrammetry, remote sensing and spatial information sciences, vol XXXVIII-3A, pp 103–108Google Scholar
  31. Shotton J, Winn J, Rother C, Criminisi A (2009) TextonBoost for image understanding: multi-class object recognition and segmentation by jointly modeling texture, layout, and context. Int J Comput Vis 81(1):2–23Google Scholar
  32. Tokarczyk P, Montoya J, Schindler K (2012) An evaluation of feature learning methods for high resolution image classification. I-3:389–394Google Scholar
  33. Valentin JP, Sengupta S, Warrell J, Shahrokni A, Torr PH (2013) Mesh based semantic modelling for indoor and outdoor scenes. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), pp 2067–2074Google Scholar
  34. Weinmann M, Jutzi B, Hinz S, Mallet C (2015a) Semantic point cloud interpretation based on optimal neighbourhoods, relevant features and efficient classifiers. ISPRS J Photogramm Remote Sens 105:286–304Google Scholar
  35. Weinmann M, Schmidt A, Mallet C, Rottensteiner F, Jutzi B (2015b) Contextual classification of point cloud data by exploiting individual 3D neighbourhoods. In: ISPRS annals of the photogrammetry, remote sensing and spatial information sciences, vol II-3/W4, pp 271–278Google Scholar
  36. Xiong X, Munoz D, Bagnell JAD, Hebert M (2011) 3-d scene analysis via sequenced predictions over points and regions. In: IEEE international conference on robotics and automation (ICRA), pp 2609–2616Google Scholar
  37. Zhou Y, Yu Y, Lu G, Du S (2012) Super-segments based classification of 3d urban street scenes. Int J Adv Robot Syst 9:1–8CrossRefGoogle Scholar

Copyright information

© Deutsche Gesellschaft für Photogrammetrie, Fernerkundung und Geoinformation (DGPF) e.V. 2017

Authors and Affiliations

  • Max Coenen
    • 1
    Email author
  • Franz Rottensteiner
    • 1
  • Christian Heipke
    • 1
  1. 1.Institut für Photogrammetrie und GeoInformationLeibniz Universität HannoverHannoverGermany

Personalised recommendations