Machine Vision and Applications

, Volume 28, Issue 1–2, pp 101–115 | Cite as

An automatic image-to-DEM alignment approach for annotating mountains pictures on a smartphone

  • Lorenzo PorziEmail author
  • Samuel Rota Bulò
  • Oswald Lanz
  • Paolo Valigi
  • Elisa Ricci
Original Paper


Augmented reality (AR) technologies enhance users knowledge of their immediate surroundings by presenting contextualized and spatially relevant information. This augmentation is enabled by the automatic estimation of the device’s pose with respect to the environment. Many simple AR applications for smartphones are well served by the pose estimate provided by GPS and inertial sensors, while other demand an higher accuracy which can be obtained with vision-based methods. The application presented in this paper, which aims to augment pictures of mountainous landscapes with geo-referenced data, belongs to the latter category. Our application is based on a novel approach for image-to-world registration which jointly relies on inertial and visual sensors. In a nutshell, first GPS and inertial sensors are used to compute a rough estimate of the device position and pose, then visual data are employed to refine it. Specifically, a learning-based edge detection algorithm is used to extract mountain profiles from the picture of interest. Then, a novel registration algorithm based on a robust optimization framework is employed to align the computed profiles to synthetic ones obtained from Digital Elevation Models. Our experiments, conducted on a novel dataset of manually aligned pictures which is made publicly available, demonstrate that the proposed registration method guarantees an increased accuracy with respect to competing methods and it is computationally efficient when implemented on a smartphone.


Augmented reality Inertial sensors Image annotation Edge detection Smartphones 


  1. 1.
    Baboud, L., Čadík, M., Eisemann, E., Seidel, H.P.: Automatic photo-to-terrain alignment for the annotation of mountain pictures. In: Proceedings of the Conference on Computer Vision and Pattern Recognition (CVPR’11), pp. 41–48 (2011)Google Scholar
  2. 2.
    Chippendale, P., Zanin, M., Andreatta, C.: Spatial and temporal attractiveness analysis through geo-referenced photo alignment. In: International Symposium on Geoscience and Remote Sensing (IGARS’08), vol. 2, pp. 1116–1119 (2008)Google Scholar
  3. 3.
    Dollár, P., Tu, Z., Belongie, S.: Supervised learning of edges and object boundaries. In: Proceedings of the Conference on Computer Vision and Pattern Recognition (CVPR’06), vol. 2, pp. 1964–1971 (2006)Google Scholar
  4. 4.
    Dollár, P., Zitnick, C.L.: Structured forests for fast edge detection. In: Proceedings of the Conference on Computer Vision (ICCV’13), pp. 1841–1848 (2013)Google Scholar
  5. 5.
    Ozuysal, M., Calonder, M., Lepetit, V., Fua, P.: Fast keypoint recognition using random ferns. IEEE Trans. Pattern Anal. Mach. Intell. 32(3), 448–461 (2010)CrossRefGoogle Scholar
  6. 6.
    Porzi, L., Rota Buló, S., Valigi, P., Lanz, O., Ricci, E.: Learning contours for automatic annotations of mountains pictures on a smartphone. In: Proceedings of the International Conference on Distributed Smart Cameras (ICDSC’14), pp. 13:1–13:6 (2014)Google Scholar
  7. 7.
  8. 8.
    Hernández, C.: Lens blur in the new google camera app.
  9. 9.
    Omerčević, D., Leonardis, A.: Hyperlinking reality via camera phones. Mach. Vis. Appl. 22(3), 521–534 (2011)Google Scholar
  10. 10.
  11. 11.
  12. 12.
  13. 13.
    Cain, N.: Showmehills ar mountain peaks.
  14. 14.
    Baatz, G., Saurer, O., Köser, K., Pollefeys, M.: Large scale visual geo-localization of images in mountainous terrain. In: Proceedings of the 12th European Conference on Computer Vision (ECCV’97), pp. 517–530 (2012)Google Scholar
  15. 15.
    Behringer, R.: Registration for outdoor augmented reality applications using computer vision techniques and hybrid sensors. In: Proceedings of the Conference on Virtual Reality, pp. 244–251 (1999)Google Scholar
  16. 16.
    Ruzon, M.A., Tomasi, C.: Color edge detection with the compass operator. In: Proceedings of the Conference on Computer Vision and Pattern Recognition (CVPR’99), vol. 2 (1999)Google Scholar
  17. 17.
    Prasad, M., Zisserman, A., Fitzgibbon, A., Kumar, M.P., Torr, P.H.S.: Learning class-specific edges for object detection and segmentation. In: Computer Vision Graphics and Image Processing, pp. 94–105 (2006)Google Scholar
  18. 18.
    Xiaofeng, R., Bo, L.: Discriminatively trained sparse code gradients for contour detection. In: Proceedings of the Advances in Neural Information Processing Systems, pp. 584–592 (2012)Google Scholar
  19. 19.
    Chekhlov, D., Pupilli, M., Mayol, W., Calway, A.: Robust real-time visual slam using scale prediction and exemplar based feature description. In: Proceedings of the Conference on Computer Vision and Pattern Recognition (CVPR’07), pp. 1–7 (2007)Google Scholar
  20. 20.
    Clemente, L.A., Davison, A.J., Reid, I.D., Neira, J., Tardós, J.D.: Mapping large loops with a single hand-held camera. In: Robotics: Science and Systems, vol. 2, p. 11 (2007)Google Scholar
  21. 21.
    Davison, A.J., Reid, I.D., Molton, N.D., Stasse, O.: Monoslam: real-time single camera slam. IEEE Trans. Pattern Anal. Mach. Intell. 29(6), 1052–1067 (2007)CrossRefGoogle Scholar
  22. 22.
    Klein, G., Murray, D.: Parallel tracking and mapping on a camera phone. In: Proceedings of the International Symposium on Mixed and Augmented Reality (ISMAR’09) (2009)Google Scholar
  23. 23.
    Schops, T., Enge, J., Cremers, D.: Semi-dense visual odometry for ar on a smartphone. In: Proceedings of the International Symposium on Mixed and Augmented Reality (ISMAR’14) (2014)Google Scholar
  24. 24.
    Ventura, J., Arth, C., Reitmayr, G., Schmalstieg, D.: Global localization from monocular slam on a mobile phone. IEEE Trans. Vis. Comput. Graph. 20, 531–539 (2014)CrossRefGoogle Scholar
  25. 25.
    Gil, A., Mozos, O.M., Ballesta, M., Reinoso, O.: A comparative evaluation of interest point detectors and local descriptors for visual slam. Mach. Vis. Appl. 21(6), 905–920 (2010)CrossRefGoogle Scholar
  26. 26.
    Cozman, F., Krotkov, E.: Position estimation from outdoor visual landmarks for teleoperation of lunar rovers. In: Proceedings of the 3rd IEEE Workshop on Applications of Computer Vision (WACV’96), pp. 156–161 (1996)Google Scholar
  27. 27.
    Dumble, S.J., Gibbens, P.W.: Efficient terrain-aided visual horizon based attitude estimation and localization. J. Intell. Robot. Syst. 2, 1–17 (2014)Google Scholar
  28. 28.
    Sim, D.G., Park, R.H.: Localization based on dem matching using multiple aerial image pairs. IEEE Trans. Image Process. 11(1), 52–55 (2002)Google Scholar
  29. 29.
    Stein, F., Medioni, G.: Map-based localization using the panoramic horizon. In: Proceedings of the International Conference on Robotics and Automation (ICRA’92), pp. 2631–2637 (1992)Google Scholar
  30. 30.
    Talluri, R., Aggarwal, J.K.: Position estimation for an autonomous mobile robot in an outdoor environment. IEEE Trans. Robot. Autom. 8(5), 573–584 (1992)CrossRefGoogle Scholar
  31. 31.
    Woo, J., Son, K., Li, T., Kim, G.S., Kweon, I.S.: Vision-based uav navigation in mountain area. In: Proceedings of the IAPR Conference on Machine Vision Applications (MVA’07), pp. 236–239 (2007)Google Scholar
  32. 32.
  33. 33.
    Ozuysal, M., Fua, P., Lepetit, V.: Fast keypoint recognition in ten lines of code. In: Proceedings Conference on Computer Vision and Pattern Recognition (CVPR’07), pp. 1–8 (2007)Google Scholar
  34. 34.
    Crow, F.C.: Summed-area tables for texture mapping. ACM SIGGRAPH Comput. Graph. 18(3), 207–212 (1984)CrossRefGoogle Scholar
  35. 35.
    Jarvis, A., Reuter, H.I., Nelson, A., Guevara, E.: Hole-filled seamless srtm data v4.
  36. 36.
    de Ferranti, J.: Viewfinder panoramas digital elevation data.
  37. 37.
    Lourenço, A., Rota Bulò, S., Fred, A., Pelillo, M.: Consensus clustering with robust evidence accumulation. In: Energy Minimization Methods in Computer Vision and Pattern Recognition. Lecture Notes in Computer Science, vol. 8081, pp. 307–320 (2013)Google Scholar
  38. 38.
    Agarwal, S., Mierle, K., et al.: Ceres solver.
  39. 39.
    Besl, P.J., McKay, N.D.: A method for registration of 3-D shapes. IEEE Trans. Pattern Anal. Mach. Intell. 14(2), 239–256 (1992)Google Scholar
  40. 40.
  41. 41.
    Huynh, D.Q.: Metrics for 3d rotations: comparison and analysis. J. Math. Imaging Vis. 35(2), 155–164 (2009)MathSciNetCrossRefGoogle Scholar
  42. 42.
    Dollár, P.: Piotr’s computer vision matlab toolbox (PMT).
  43. 43.
    Canny, J.: A computational approach to edge detection. IEEE Trans. Pattern Anal. Mach. Intell. 6, 679–698 (1986)CrossRefGoogle Scholar
  44. 44.
    Yang, X.S., Deb, S., Fong, S.: Accelerated particle swarm optimization and support vector machine for business optimization and applications. In: Networked Digital Technologies, pp. 53–66 (2011)Google Scholar
  45. 45.
    Porzi, L., Ricci, E., Ciarfuglia, T.A., Zanin, M.: Visual-inertial tracking on android for augmented reality applications. In: Proceedings of the Workshop on Environmental, Energy and Structural Monitoring Systems (EESMS’12), pp. 35–41 (2012)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2016

Authors and Affiliations

  1. 1.Department of EngineeringUniversity of PerugiaPerugiaItaly
  2. 2.Fondazione Bruno KesslerTrentoItaly

Personalised recommendations