Skip to main content
SpringerLink
Log in

Classification of Stereo Images from Mobile Mapping Data Using Conditional Random Fields

  • Original Article
  • Published:
PFG – Journal of Photogrammetry, Remote Sensing and Geoinformation Science Aims and scope Submit manuscript

Abstract

We propose a new method for the context-based classification of point clouds from stereo images using Conditional Random Fields (CRF). The classification is based on segments as nodes for the CRF. The segmentation is conducted on the image and is transferred to the 3D point cloud obtained by image matching. This allows the computation of 3D features additionally to the image features as well as the definition of realistic adjacencies between the segments in object space. We also propose a variant of the contrast-sensitive Potts model that is tailored for the contextual classification of point cloud segments. The evaluation of our method is performed on stereo sequences of a benchmark dataset, recorded in an urban area, and yields results with an overall accuracy of more than 90%. Moreover, we can show that the consideration of contextual information during the classification leads to an improvement of the overall accuracy.

Zusammenfassung

Klassifikation von Stereobildern aus Mobile Mapping Daten mittels Conditional Random Fields. In dieser Arbeit wird ein neues Verfahren zur kontextbasierten Klassifikation von Punktwolken aus Stereobildern mittels Conditional Random Fields (CRF) vorgestellt. Die Klassifikation setzt auf Segmenten als Knoten für das CRF auf. Die Segmentierung erfolgt im Bildraum und wird mittels einer 3D-Rekonstruktion der Szene auf die 3D-Punktwolke übertragen, was die Extraktion von 3D-Merkmalen zusätzlich zu den Bildmerkmalen sowie die Definition von realistischen Nachbarschaftsbeziehungen zwischen den Segmenten im Objektraum ermöglicht. Außerdem wird eine Variante des kontrastsensitiven Potts-Modells vorgestellt, welches für die kontextbasierte Klassifikation von Punktwolkensegmenten maßgeschneidert ist. Die Evaluierung der Methode erfolgt anhand von im urbanen Raum aufgenommenen Stereosequenzen eines Benchmark Datensatzes und liefert Ergebnisse mit einer Gesamtgenauigkeit von über 90%. Außerdem wird gezeigt, dass die Berücksichtigung von Kontext in der Klassifikation zu einer Erhöhung der Gesamtgenauigkeiten führt.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

References

  • Achanta R, Shaji A, Smith K, Lucchi A, Fua P, Snsstrunk S (2012) SLIC superpixels compared to state-of-the-art superpixel methods. IEEE Trans Pattern Anal Mach Intell 34(11):2274–2282

    Article  Google Scholar 

  • Aijazi AK, Checchin P, Trassoudaine L (2013) Segmentation based classification of 3d urban point clouds: a super-voxel based approach with evaluation. Remote Sens 5(4):1624–1650

  • Anguelov D, Taskarf B, Chatalbashev V, Koller D, Gupta D, Heitz G, Ng A (2005) Discriminative learning of Markov random fields for segmentation of 3d scan data. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), vol 2, pp 169–176

  • Bishop CM (2006) Pattern recognition and machine learning, 1st edn. Springer. New York

  • Breiman L (2001) Random forests. Mach Learn 45(1):5–32

    Article  Google Scholar 

  • Brostow GJ, Shotton J, Fauqueur J, Cipolla R (2008) Segmentation and recognition using structure from motion point clouds. In: Proceedings of the european conference on computer vision (ECCV), vol. I, pp 44–57

  • Chehata N, Guo L, Mallet C (2009) Airborne lidar feature selection for urban classification using random forests. In: International archives of the photogrammetry, remote sensing and spatial information sciences, vol XXXVIII-3/W8, pp 207–212

  • Dohan D, Matejek B, Funkhouser T (2015) Learning hierarchical semantic segmentations of LIDAR data. In: International conference on 3D vision, pp 273–281

  • Frey B, MacKay D (1998) A revolution: belief propagation in graphs with cycles. Adv Neur Inf Process Syst 10:479–485

    Google Scholar 

  • Geiger A, Lenz P, Urtasun R (2012) Are we ready for autonomous driving? The KITTI vision benchmark suite. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), pp 3354–3361

  • Geiger A, Roser M, Urtasun R (2011) Efficient large-scale stereo matching. In: Proceedings of the 10th Asian conference on computer vision, vol I, pp 25–38

  • Geman S, Geman D (1984) Stochastic relaxation, Gibbs distributions, and the Bayesian restoration of images. IEEE Trans Pattern Anal Mach Intell 6(6):721–741

    Article  Google Scholar 

  • Hackel T, Wegner JD, Schindler K (2016) Fast semantic segmentation of 3d point clouds with strongly varying density. In: ISPRS annals of the photogrammetry, remote sensing and spatial information sciences, vol III-3, pp 177–184

  • Haralick RM, Shanmugam K, Dinstein I (1973) Textural features for image classification. IEEE Trans Syst Man Cybern SMC-3(6):610–621

  • Heipke C, Mayer H, Wiedemann C, Jamet O (1997) Evaluation of automatic road extraction. In: International archives of photogrammetry and remote sensing, vol XXXII-3, pp 151–160

  • Kang Y, Yamaguchi K, Naito T, Ninomiya Y (2009) Road scene labeling using SfM module and 3D bag of textons. In: 12th international conference on computer vision workshops (ICCV workshops), pp 657–664

  • Kumar S, Hebert M (2006) Discrimantive random fields. Int J Comput Vis 68(2):179–201

    Article  Google Scholar 

  • Li X, Sahbi H (2011) Superpixel-based object class segmentation using conditional random fields. In: International conference on acoustics, speech and signal processing (ICASSP), pp 1101–1104

  • Lim EH, Suter D (2009) 3D terrestrial LIDAR classifications with super-voxels and multi-scale conditional random fields. Comput-Aided Des 41(10):701–710

    Article  Google Scholar 

  • Lucchi A, Li Y, Smith K, Fua P (2012) Structured image segmentation using kernelized features. In: Proceedings of the European conference on computer vision (ECCV), vol II, pp 400–413

  • Matti EK, Nebiker S (2014) Geometry and colour based classification of urban point cloud scenes using a supervised self-organizing map. Photogrammetrie Fernerkundung Geoinformation 3:161–173

  • Munoz D, Bagnell JA, Hebert M (2012) Co-inference for multi-modal scene analysis. In: Proceedings of the European conference on computer vision (ECCV), vol VI, pp 668–681

  • Munoz D, Vandapel N, Hebert M (2009) Onboard contextual classification of 3-D point clouds with learned high-order Markov random fields. In: IEEE international conference on robotics and automation ICRA ’09, pp 2009–2016

  • Najafi M, Namin S, Salzmann M, Petersson L (2014) Non-associative higher-order markov networks for point cloud classification. In: Proceedings of the european conference on computer vision (ECCV), pp 500–515

  • Niemeyer J, Rottensteiner F, Soergel U (2012) Conditional random fields for lidar point cloud classification in complex urban areas. In: ISPRS annals of the photogrammetry, remote sensing and spatial information sciences, vol I-3, pp 263–268

  • Niemeyer J, Rottensteiner F, Sörgel U (2014) Contextual classification of Lidar data and building object detection in urban areas. ISPRS J Photogramm Remote Sens 87(2014):152–165

    Article  Google Scholar 

  • Reich M, Unger J, Rottensteiner F, Heipke C (2013) On-line compatible orientation of a micro-UAV based on image triplets. In: ISPRS annals of the photogrammetry, remote sensing and spatial information sciences, vol II-3/W2, pp 37–42

  • Rusu RB, Holzbach A, Blodow N, Beetz M (2009) Fast geometric point labeling using conditional random fields. In: International conference on intelligent robots and systems, pp 7–12

  • Sengupta S, Greverson E, Shahrokni A, Torr P (2013) Urban 3D semantic modelling using stereo vision. In: IEEE international conference on robotics and automation, pp 580–585

  • Shapovalov R, Velizhev E, Barinova O (2010) Non-associative Markov networks for 3d point cloud classification. In: International archives of the photogrammetry, remote sensing and spatial information sciences, vol XXXVIII-3A, pp 103–108

  • Shotton J, Winn J, Rother C, Criminisi A (2009) TextonBoost for image understanding: multi-class object recognition and segmentation by jointly modeling texture, layout, and context. Int J Comput Vis 81(1):2–23

  • Tokarczyk P, Montoya J, Schindler K (2012) An evaluation of feature learning methods for high resolution image classification. I-3:389–394

  • Valentin JP, Sengupta S, Warrell J, Shahrokni A, Torr PH (2013) Mesh based semantic modelling for indoor and outdoor scenes. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), pp 2067–2074

  • Weinmann M, Jutzi B, Hinz S, Mallet C (2015a) Semantic point cloud interpretation based on optimal neighbourhoods, relevant features and efficient classifiers. ISPRS J Photogramm Remote Sens 105:286–304

  • Weinmann M, Schmidt A, Mallet C, Rottensteiner F, Jutzi B (2015b) Contextual classification of point cloud data by exploiting individual 3D neighbourhoods. In: ISPRS annals of the photogrammetry, remote sensing and spatial information sciences, vol II-3/W4, pp 271–278

  • Xiong X, Munoz D, Bagnell JAD, Hebert M (2011) 3-d scene analysis via sequenced predictions over points and regions. In: IEEE international conference on robotics and automation (ICRA), pp 2609–2616

  • Zhou Y, Yu Y, Lu G, Du S (2012) Super-segments based classification of 3d urban street scenes. Int J Adv Robot Syst 9:1–8

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Institut für Photogrammetrie und GeoInformation, Leibniz Universität Hannover, 30167, Hannover, Germany

    Max Coenen, Franz Rottensteiner & Christian Heipke

Authors
  1. Max Coenen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Franz Rottensteiner
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Christian Heipke
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Max Coenen.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Coenen, M., Rottensteiner, F. & Heipke, C. Classification of Stereo Images from Mobile Mapping Data Using Conditional Random Fields. PFG 85, 17–30 (2017). https://doi.org/10.1007/s41064-017-0004-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s41064-017-0004-5

Keywords

Search

Navigation