Skip to main content

The World Is Changing: Finding Changes on the Street

  • 1757 Accesses

Part of the Lecture Notes in Computer Science book series (LNIP,volume 10116)


We propose to find changes in the constantly changing world, given visual observations at street-level. In particular, we identify “long-term” changes between Google Street View images and dashcam videos captured at different months or even years. This is a challenging task, since (1) dashcam frames are not localized in world coordinate, and (2) there are many changes introduced by moving objects. We propose a robust sequence alignment method to align dashcam sequence to Street View images. Our method outperforms a strong baseline method [1] by \(12\%\) mean Average Precision (AP). We also propose a novel change detection method designed to detect long-term changes. Our change detection method (\({13.54}\%\)) outperforms a baseline method without handling car interior and moving objects (\({11.70}\%\)) by \(1.84\%\) (relatively \(13.6\%\)) in mean AP. In a controlled experiment, given manually aligned high quality Street View images, our change detection method achieves a significantly better mean AP (\(45.57\%\)).


  • Reference Image
  • Average Precision
  • Reconstruction Error
  • Query Image
  • Confidence Score

These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

This is a preview of subscription content, access via your institution.

Buying options

USD   29.95
Price excludes VAT (USA)
  • DOI: 10.1007/978-3-319-54407-6_28
  • Chapter length: 16 pages
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
USD   84.99
Price excludes VAT (USA)
  • ISBN: 978-3-319-54407-6
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
Softcover Book
USD   109.99
Price excludes VAT (USA)
Fig. 1.
Fig. 2.
Fig. 3.
Fig. 4.
Fig. 5.
Fig. 6.
Fig. 7.
Fig. 8.
Fig. 9.
Fig. 10.
Fig. 11.


  1. 1.

    It is very common that the rough GPS location of a dashcam video is described in the video description for the purpose of reporting accident.

  2. 2.

    We calculate the intersection over union area.

  3. 3.

    \(30\%\) is used, since ground truth changes are typical irregular and possibly consists of more than one objects. It is very challenging to precisely detect the ground truth change rectangle.


  1. Vaca-Castano, G., Zamir, A., Shah, M.: City scale geo-spatial trajectory estimation of a moving camera. In: CVPR (2012)

    Google Scholar 

  2. Badino, H., Huber, D., Kanade, T.: Real-time topometric localization. In: ICRA (2012)

    Google Scholar 

  3. Kim, J., Liu, C., Sha, F., Grauman, K.: Deformable spatial pyramid matching for fast dense correspondences. In: CVPR (2013)

    Google Scholar 

  4. Robertson, D., Cipolla, R.: An image-based system for urban navigation. In: BMVC (2004)

    Google Scholar 

  5. Zhang, W., Kosecka, J.: Image based localization in urban environments. In: 3DPVT (2006)

    Google Scholar 

  6. Schindler, G., Brown, M., Szeliski, R.: City-scale location recognition. In: CVPR (2007)

    Google Scholar 

  7. Hays, J., Efros, A.A.: Im2gps: estimating geographic information from a single image. In: CVPR (2008)

    Google Scholar 

  8. Zamir, A.R., Shah, M.: Accurate image localization based on google maps street view. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010. LNCS, vol. 6314, pp. 255–268. Springer, Heidelberg (2010). doi:10.1007/978-3-642-15561-1_19

    CrossRef  Google Scholar 

  9. Cao, S., Snavely, N.: Graph-based discriminative learning for location recognition. In: CVPR (2013)

    Google Scholar 

  10. Bettadapura, V., Essa, I., Pantofaru, C.: Egocentric field-of-view localization using first-person point-of-view devices. In: WACV (2015)

    Google Scholar 

  11. Irschara, A., Zach, C., Frahm, J., Bischof, H.: From structure-from-motion point clouds to fast location recognition. In: CVPR (2009)

    Google Scholar 

  12. Li, Y., Snavely, N., Huttenlocher, D.P.: Location recognition using prioritized feature matching. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010. LNCS, vol. 6312, pp. 791–804. Springer, Heidelberg (2010). doi:10.1007/978-3-642-15552-9_57

    CrossRef  Google Scholar 

  13. Sattler, T., Leibe, B., Kobbelt, L.: Fast image-based localization using direct 2d-to-3d matching. In: ICCV (2011)

    Google Scholar 

  14. Li, Y., Snavely, N., Huttenlocher, D., Fua, P.: Worldwide pose estimation using 3D point clouds. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds.) ECCV 2012. LNCS, vol. 7572, pp. 15–29. Springer, Heidelberg (2012). doi:10.1007/978-3-642-33718-5_2

    CrossRef  Google Scholar 

  15. Taneja, A., Ballan, L., Pollefeys, M.: Never get lost again: vision based navigation using streetview images. In: Cremers, D., Reid, I., Saito, H., Yang, M.-H. (eds.) ACCV 2014. LNCS, vol. 9007, pp. 99–114. Springer, Heidelberg (2015). doi:10.1007/978-3-319-16814-2_7

    Google Scholar 

  16. Lategahn, H., Schreiber, M., Ziegler, J., Stiller, C.: Urban localization with camera and inertial measurement unit. In: Intelligent Vehicles Symposium (IV) (2013)

    Google Scholar 

  17. Floros, G., van der Zander, B., Leibe, B.: OpenStreetSLAM: global vehicle localization using openstreetmaps. In: ICRA (2013)

    Google Scholar 

  18. Brubaker, M., Geiger, A., Urtasun, R.: Lost! leveraging the crowd for probabilistic visual self-localization. In: CVPR (2013)

    Google Scholar 

  19. Radke, R., Andra, S., Al-Kofahi, O., Roysam, B.: Image change detection algorithms: a systematic survey. TIP 14, 294–307 (2005)

    MathSciNet  Google Scholar 

  20. Pollard, T., Mundy, J.: Change detection in a 3-d world. In: CVPR (2007)

    Google Scholar 

  21. Taneja, A., Ballan, L., Pollefeys, M.: City-scale change detection in cadastral 3d models using images. In: CVPR (2013)

    Google Scholar 

  22. Matzen, K., Snavely, N.: Scene chronology. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8695, pp. 615–630. Springer, Heidelberg (2014). doi:10.1007/978-3-319-10584-0_40

    Google Scholar 

  23. Wu, C.: Towards linear-time incremental structure from motion. In: 3DV (2013)

    Google Scholar 

  24. Wu, C.: Visualsfm: a visual structure from motion system.

  25. Lowe, D.G.: Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vis. 60, 91–110 (2004)

    CrossRef  Google Scholar 

  26. Jegou, H., Perronnin, F., Douze, M., Snchez, J., Perez, P., Schmid, C.: Aggregating local image descriptors into compact codes. TPAMI 34, 1704–1716 (2011)

    CrossRef  Google Scholar 

  27. Zheng, S., Jayasumana, S., Romera-Paredes, B., Vineet, V., Su, Z., Du, D., Huang, C., Torr, P.: Conditional random fields as recurrent neural networks. In: ICCV (2015)

    Google Scholar 

  28. Long, J., Shelhamer, E., Darrell, T.: Fully convolutional networks for semantic segmentation. In: CVPR (2015)

    Google Scholar 

  29. Zitnick, C.L., Dollár, P.: Edge boxes: locating object proposals from edges. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8693, pp. 391–405. Springer, Heidelberg (2014). doi:10.1007/978-3-319-10602-1_26

    Google Scholar 

Download references


We thank Industrial Technology Research Institute (ITRI) project grants and MOST 103-2218-E-007-025 and MOST 104-3115-E-007-005 in Taiwan for their support.

Author information

Authors and Affiliations


Corresponding author

Correspondence to Min Sun .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and Permissions

Copyright information

© 2017 Springer International Publishing AG

About this paper

Cite this paper

Chen, KT., Wang, FE., Lin, JT., Chan, FH., Sun, M. (2017). The World Is Changing: Finding Changes on the Street. In: Chen, CS., Lu, J., Ma, KK. (eds) Computer Vision – ACCV 2016 Workshops. ACCV 2016. Lecture Notes in Computer Science(), vol 10116. Springer, Cham.

Download citation

  • DOI:

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-54406-9

  • Online ISBN: 978-3-319-54407-6

  • eBook Packages: Computer ScienceComputer Science (R0)