A Dataset for Lane Instance Segmentation in Urban Environments

  • Brook Roberts
  • Sebastian KaltwangEmail author
  • Sina Samangooei
  • Mark Pender-Bare
  • Konstantinos Tertikas
  • John Redford
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11212)


Autonomous vehicles require knowledge of the surrounding road layout, which can be predicted by state-of-the-art CNNs. This work addresses the current lack of data for determining lane instances, which are needed for various driving manoeuvres. The main issue is the time-consuming manual labelling process, typically applied per image. We notice that driving the car is itself a form of annotation. Therefore, we propose a semi-automated method that allows for efficient labelling of image sequences by utilising an estimated road plane in 3D based on where the car has driven and projecting labels from this plane into all images of the sequence. The average labelling time per image is reduced to 5 s and only an inexpensive dash-cam is required for data capture. We are releasing a dataset of 24,000 images and additionally show experimental semantic segmentation and instance segmentation results.


Dataset Urban driving Road Lane Instance segmentation Semi-automated annotation Partial labels 



We would like to thank our colleagues Tom Westmacott, Joel Jakubovic and Robert Chandler, who have contributed to the implementation of the annotation software.

Supplementary material

474213_1_En_33_MOESM1_ESM.mp4 (7.5 mb)
Supplementary material 1 (mp4 7653 KB)

Supplementary material 2 (mp4 21079 KB)

474213_1_En_33_MOESM3_ESM.mp4 (2.5 mb)
Supplementary material 3 (mp4 2516 KB)


  1. 1.
    Janai, J., Güney, F., Behl, A., Geiger, A.: Computer Vision for Autonomous Vehicles: Problems, Datasets and State-of-the-Art (2017)Google Scholar
  2. 2.
    Huval, B., et al.: An empirical evaluation of deep learning on highway driving (2015). arXiv preprint arXiv:1504.01716
  3. 3.
    Oliveira, G.L., Burgard, W., Brox, T.: Efficient deep methods for monocular road segmentation. In: IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2016), pp. 4885–4891 (2016)Google Scholar
  4. 4.
    Neven, D., De Brabandere, B., Georgoulis, S., Proesmans, M., Van Gool, L.: Towards End-to-End Lane Detection: an Instance Segmentation Approach (2018). arXiv preprint arXiv:1802.05591
  5. 5.
    Brostow, G.J., Fauqueur, J., Cipolla, R.: Semantic object classes in video: a high-definition ground truth database. Pattern Recognit. Lett. 30(2), 88–97 (2009)CrossRefGoogle Scholar
  6. 6.
    Cordts, M., et al.: The cityscapes dataset for semantic urban scene understanding. In: CVPR (2016)Google Scholar
  7. 7.
    Neuhold, G., Ollmann, T., Bulò, S.R., Kontschieder, P.: The mapillary vistas dataset for semantic understanding of street scenes. In: Proceedings of the International Conference on Computer Vision (ICCV), Venice, pp. 22–29 (2017)Google Scholar
  8. 8.
    McCall, J.C., Trivedi, M.M.: Video-based lane estimation and tracking for driver assistance: survey, system, and evaluation. IEEE Trans. Intell. Transp. Syst. 7(1), 20–37 (2006)CrossRefGoogle Scholar
  9. 9.
    Kim, Z.: Robust lane detection and tracking in challenging scenarios. IEEE Trans. Intell. Transp. Syst. 9(1), 16–26 (2008)CrossRefGoogle Scholar
  10. 10.
    Gopalan, R., Hong, T., Shneier, M., Chellappa, R.: A learning approach towards detection and tracking of lane markings. IEEE Trans. Intell. Transp. Syst. 13(3), 1088–1098 (2012)CrossRefGoogle Scholar
  11. 11.
    Li, J., Mei, X., Prokhorov, D., Tao, D.: Deep neural network for structural prediction and lane detection in traffic scene. IEEE Trans. Neural Netw. Learn. Syst. 28(3), 690–703 (2017)CrossRefGoogle Scholar
  12. 12.
    Mathibela, B., Newman, P., Posner, I.: Reading the road: road marking classification and interpretation. IEEE Trans. Intell. Transp. Syst. 16(4), 2072–2081 (2015)CrossRefGoogle Scholar
  13. 13.
    Hillel, A.B., Lerner, R., Levi, D., Raz, G.: Recent progress in road and lane detection: a survey. Mach. Vis. Appl. 25(3), 727–745 (2014)CrossRefGoogle Scholar
  14. 14.
    Brostow, G.J., Shotton, J., Fauqueur, J., Cipolla, R.: Segmentation and recognition using structure from motion point clouds. In: Forsyth, D., Torr, P., Zisserman, A. (eds.) ECCV 2008. LNCS, vol. 5302, pp. 44–57. Springer, Heidelberg (2008). Scholar
  15. 15.
    Sengupta, S., Sturgess, P., Torr, P.H.S., et al.: Automatic dense visual semantic mapping from street-level imagery. In: IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 857–862. IEEE (2012)Google Scholar
  16. 16.
    Scharwächter, T., Enzweiler, M., Franke, U., Roth, S.: Efficient multi-cue scene segmentation. In: Weickert, J., Hein, M., Schiele, B. (eds.) GCPR 2013. LNCS, vol. 8142, pp. 435–445. Springer, Heidelberg (2013). Scholar
  17. 17.
    Matzen, K., Snavely, N.: NYC3DCars: a dataset of 3D vehicles in geographic context. In: ICCV, pp. 761–768. IEEE (2013)Google Scholar
  18. 18.
    Fritsch, J., Kuehnl, T., Geiger, A.: A new performance measure and evaluation benchmark for road detection algorithms. In: 16th International IEEE Conference on Intelligent Transportation Systems (ITSC 2013), pp. 1693–1700. IEEE (2013)Google Scholar
  19. 19.
    Aly, M.: Real time detection of lane markers in urban streets. In: IEEE Intelligent Vehicles Symposium, Proceedings, pp. 7–12. IEEE (2008)Google Scholar
  20. 20.
    TuSimple: Lane Detection Challenge (Dataset) (2017).
  21. 21.
    Richter, S.R., Vineet, V., Roth, S., Koltun, V.: Playing for data: ground truth from computer games. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9906, pp. 102–118. Springer, Cham (2016). Scholar
  22. 22.
    Ros, G., Sellart, L., Materzynska, J., Vazquez, D., Lopez, A.M.: The synthia dataset: a large collection of synthetic images for semantic segmentation of urban scenes. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3234–3243 (2016)Google Scholar
  23. 23.
    Gaidon, A., Wang, Q., Cabon, Y., Vig, E.: Virtual Worlds as Proxy for Multi-Object Tracking Analysis. In: CVPR (2016)Google Scholar
  24. 24.
    Leibe, B., Cornelis, N., Cornelis, K., Van Gool, L.: Dynamic 3D scene analysis from a moving vehicle. In: CVPR, pp. 1–8. IEEE (2007)Google Scholar
  25. 25.
    Borkar, A., Hayes, M., Smith, M.T.: A novel lane detection system with efficient ground truth generation. IEEE Trans. Intell. Transp. Syst. 13(1), 365–374 (2012)CrossRefGoogle Scholar
  26. 26.
    Laddha, A., Kocamaz, M.K., Navarro-Serment, L.E., Hebert, M.: Map-supervised road detection. In: IEEE Intelligent Vehicles Symposium (IV), pp. 118–123. IEEE (2016)Google Scholar
  27. 27.
    Xie, J., Kiefel, M., Sun, M.T., Geiger, A.: Semantic instance annotation of street scenes by 3D to 2D label transfer. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3688–3697 (2016)Google Scholar
  28. 28.
    Barnes, D., Maddern, W., Posner, I.: Find your own way: weakly-supervised segmentation of path proposals for urban autonomy. In: ICRA (2017)Google Scholar
  29. 29.
    Mapillary: OpenSfM (Software) (2014).
  30. 30.
    Badrinarayanan, V., Kendall, A., Cipolla, R.: Segnet: A deep convolutional encoder-decoder architecture for image segmentation (2015). arXiv preprint arXiv:1511.00561
  31. 31.
    Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. CoRR (2014)Google Scholar
  32. 32.
    Caesar, H., Uijlings, J., Ferrari, V.: Coco-stuff: Thing and Stuff Classes in Context (2017). arXiv preprint. arXiv:1612.03716
  33. 33.
    Adelson, E.H.: On seeing stuff: the perception of materials by humans and machines. In: Rogowitz, B.E., Pappas, T.N. (Eds.): Society of Photo-Optical Instrumentation Engineers (SPIE) Conference Series, vol. 4299, pp. 1–12, June 2001Google Scholar
  34. 34.
    He, K., Gkioxari, G., Dollár, P., Girshick, R.B.: Mask R-CNN. CoRR (2017)Google Scholar
  35. 35.
    Brabandere, B.D., Neven, D., Gool, L.V.: Semantic instance segmentation with a discriminative loss function. CoRR (2017)Google Scholar
  36. 36.
    Li, S., Seybold, B., Vorobyov, A., Fathi, A., Huang, Q., Kuo, C.C.J.: Instance embedding transfer to unsupervised video object segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 6526–6535 (2018)Google Scholar
  37. 37.
    Fathi, A., Wojna, Z., Rathod, V., Wang, P., Song, H.O., Guadarrama, S., Murphy, K.P.: Semantic instance segmentation via deep metric learning. CoRR (2017)Google Scholar
  38. 38.
    Kong, S., Fowlkes, C.: Recurrent pixel embedding for instance grouping (2017)Google Scholar
  39. 39.
    Comaniciu, D., Meer, P.: Mean shift: a robust approach toward feature space analysis. IEEE Trans. Pattern Anal. Mach. Intell. 24(5), 603–619 (2002)CrossRefGoogle Scholar
  40. 40.
    Lin, T.-Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., Zitnick, C.L.: Microsoft COCO: common objects in context. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8693, pp. 740–755. Springer, Cham (2014). Scholar

Copyright information

© Springer Nature Switzerland AG 2018

Authors and Affiliations

  • Brook Roberts
    • 1
  • Sebastian Kaltwang
    • 1
    Email author
  • Sina Samangooei
    • 1
  • Mark Pender-Bare
    • 1
  • Konstantinos Tertikas
    • 1
  • John Redford
    • 1
  1. 1.FiveAI Ltd.CambridgeUK

Personalised recommendations