Advertisement

Privacy Preserving Structure-from-Motion

Conference paper
  • 1.7k Downloads
Part of the Lecture Notes in Computer Science book series (LNCS, volume 12346)

Abstract

Over the last years, visual localization and mapping solutions have been adopted by an increasing number of mixed reality and robotics systems. The recent trend towards cloud-based localization and mapping systems has raised significant privacy concerns. These are mainly grounded by the fact that these services require users to upload visual data to their servers, which can reveal potentially confidential information, even if only derived image features are uploaded. Recent research addresses some of these concerns for the task of image-based localization by concealing the geometry of the query images and database maps. The core idea of the approach is to lift 2D/3D feature points to random lines, while still providing sufficient constraints for camera pose estimation. In this paper, we further build upon this idea and propose solutions to the different core algorithms of an incremental Structure-from-Motion pipeline based on random line features. With this work, we make another fundamental step towards enabling privacy preserving cloud-based mapping solutions. Various experiments on challenging real-world datasets demonstrate the practicality of our approach achieving comparable results to standard Structure-from-Motion systems.

Notes

Acknowledgements

Viktor Larsson was supported by an ETH Zurich Postdoctoral Fellowship.

Supplementary material

500725_1_En_20_MOESM1_ESM.pdf (30.8 mb)
Supplementary material 1 (pdf 31575 KB)

Supplementary material 2 (mp4 61384 KB)

References

  1. 1.
    Abadi, M., et al.: Deep learning with differential privacy. In: Conference on Computer and Communications Security (ACM CCS) (2016)Google Scholar
  2. 2.
    Agarwal, S., Mierle, K., et al.: Ceres solver. http://ceres-solver.org
  3. 3.
    Andrés, M.E., Bordenabe, N.E., Chatzikokolakis, K., Palamidessi, C.: Geo-indistinguishability: differential privacy for location-based systems. In: Conference on Computer & Communications Security (SIGSAC) (2013)Google Scholar
  4. 4.
    Aranda, M., López-Nicolás, G., Sagüés, C.: Omnidirectional visual homing using the 1D trifocal tensor. In: International Conference on Robotics and Automation (ICRA) (2010)Google Scholar
  5. 5.
    Ardagna, C.A., Cremonini, M., Damiani, E., De Capitani di Vimercati, S., Samarati, P.: Location privacy protection through obfuscation-based techniques. In: Barker, S., Ahn, G.-J. (eds.) DBSec 2007. LNCS, vol. 4602, pp. 47–60. Springer, Heidelberg (2007).  https://doi.org/10.1007/978-3-540-73538-0_4CrossRefGoogle Scholar
  6. 6.
  7. 7.
  8. 8.
    Åström, K., Oskarsson, M.: Solutions and ambiguities of the structure and motion problem for 1D retinal vision. J. Math. Imaging Vis. 12, 121–135 (2000) MathSciNetCrossRefGoogle Scholar
  9. 9.
    Avidan, S., Butman, M.: Blind vision. In: Leonardis, A., Bischof, H., Pinz, A. (eds.) ECCV 2006, Part III. LNCS, vol. 3953, pp. 1–13. Springer, Heidelberg (2006).  https://doi.org/10.1007/11744078_1CrossRefGoogle Scholar
  10. 10.
    Avidan, S., Butman, M.: Efficient methods for privacy preserving face detection. In: Advances in Neural Information Processing Systems (2007)Google Scholar
  11. 11.
    Dellaert, F., Stroupe, A.W.: Linear 2D localization and mapping for single and multiple robot scenarios. In: International Conference on Robotics and Automation (ICRA) (2002)Google Scholar
  12. 12.
    Dinur, I., Nissim, K.: Revealing information while preserving privacy. In: Symposium on Principles of Database Systems (2003)Google Scholar
  13. 13.
    Dosovitskiy, A., Brox, T.: Generating images with perceptual similarity metrics based on deep networks. In: Advances in Neural Information Processing Systems (2016)Google Scholar
  14. 14.
    Dosovitskiy, A., Brox, T.: Inverting visual representations with convolutional networks. In: Computer Vision and Pattern Recognition (CVPR) (2016)Google Scholar
  15. 15.
    Dwork, C.: Differential privacy: a survey of results. In: Agrawal, M., Du, D., Duan, Z., Li, A. (eds.) TAMC 2008. LNCS, vol. 4978, pp. 1–19. Springer, Heidelberg (2008).  https://doi.org/10.1007/978-3-540-79228-4_1CrossRefzbMATHGoogle Scholar
  16. 16.
    Erkin, Z., Franz, M., Guajardo, J., Katzenbeisser, S., Lagendijk, I., Toft, T.: Privacy-preserving face recognition. In: Goldberg, I., Atallah, M.J. (eds.) PETS 2009. LNCS, vol. 5672, pp. 235–253. Springer, Heidelberg (2009).  https://doi.org/10.1007/978-3-642-03168-7_14CrossRefGoogle Scholar
  17. 17.
    Gedik, B., Liu, L.: Protecting location privacy with personalized K-anonymity: architecture and algorithms. IEEE Trans. Mob. Comput. 7, 1–18 (2008)CrossRefGoogle Scholar
  18. 18.
    Gilad-Bachrach, R., Dowlin, N., Laine, K., Lauter, K., Naehrig, M., Wernsing, J.: Cryptonets: applying neural networks to encrypted data with high throughput and accuracy. In: International Conference on Machine Learning (2016)Google Scholar
  19. 19.
    Kato, H., Harada, T.: Image reconstruction from bag-of-visual-words. In: Computer Vision and Pattern Recognition (CVPR) (2014)Google Scholar
  20. 20.
    Kipman, A.: Azure Spatial Anchors approach to privacy and ethical design (2019). https://www.linkedin.com/pulse/azure-spatial-anchors-approach-privacy-ethical-design-alex-kipman/
  21. 21.
    Kukelova, Z., Heller, J., Fitzgibbon, A.W.: Efficient intersection of three quadrics and applications in computer vision. In: Computer Vision and Pattern Recognition (CVPR) (2016)Google Scholar
  22. 22.
    Lebeda, K., Matas, J., Chum, O.: Fixing the locally optimized RANSAC. In: British Machine Vision Conference (BMVC) (2012)Google Scholar
  23. 23.
    Mahendran, A., Vedaldi, A.: Understanding deep image representations by inverting them. In: Computer Vision and Pattern Recognition (CVPR) (2015)Google Scholar
  24. 24.
    Nielsen, M.L.: Augmented Reality and its Impact on the Internet, Security, and Privacy (2015). https://beyondstandards.ieee.org/augmented-reality/augmented-reality-and-its-impact-on-the-internet-security-and-privacy/
  25. 25.
    Pittaluga, F., Koppal, S., Chakrabarti, A.: Learning privacy preserving encodings through adversarial training. In: Winter Conference on Applications of Computer Vision (WACV) (2019)Google Scholar
  26. 26.
    Pittaluga, F., Koppal, S., Kang, S.B., Sinha, S.N.: Revealing scenes by inverting structure from motion reconstructions. In: Computer Vision and Pattern Recognition (CVPR) (2019)Google Scholar
  27. 27.
    Pittaluga, F., Koppal, S.J.: Privacy preserving optics for miniature vision sensors. In: Computer Vision and Pattern Recognition (CVPR) (2015)Google Scholar
  28. 28.
    Pittaluga, F., Koppal, S.J.: Pre-capture privacy for small vision sensors. Trans. Pattern Anal. Mach. Intell. (TPAMI) 39, 2215–2226 (2017)CrossRefGoogle Scholar
  29. 29.
    Quan, L., Kanade, T.: Affine structure from line correspondences with uncalibrated affine cameras. Trans. Pattern Anal. Mach. Intell. (TPAMI) 19, 834–845 (1997)CrossRefGoogle Scholar
  30. 30.
    Ren, Z., Lee, Y.J., Ryoo, M.S.: Learning to anonymize faces for privacy preserving action detection. In: Ferrari, V., Hebert, M., Sminchisescu, C., Weiss, Y. (eds.) ECCV 2018, Part I. LNCS, vol. 11205, pp. 639–655. Springer, Cham (2018).  https://doi.org/10.1007/978-3-030-01246-5_38CrossRefGoogle Scholar
  31. 31.
    Roesner, F.: Who Is Thinking About Security and Privacy for Augmented Reality? (2017). https://www.technologyreview.com/s/609143/who-is-thinking-about-security-and-privacy-for-augmented-reality/
  32. 32.
    Ryoo, M.S., Rothrock, B., Fleming, C., Yang, H.J.: Privacy-preserving human activity recognition from extreme low resolution. In: Proceedings of the Thirty-First Conference on Artificial Intelligence (AAAI) (2017)Google Scholar
  33. 33.
    Sagues, C., Murillo, A., Guerrero, J.J., Goedemé, T., Tuytelaars, T., Gool, L.V.: Localization with omnidirectional images using the radial trifocal tensor. In: International Conference on Robotics and Automation (ICRA) (2006)Google Scholar
  34. 34.
    Samarati, P.: Protecting respondents identities in microdata release. IEEE Trans. Knowl. Data Eng. 13, 1010–1027 (2001)CrossRefGoogle Scholar
  35. 35.
    Scape: Scape Technologies (2019). https://scape.io/
  36. 36.
    Schönberger, J.L., Frahm, J.M.: Structure-from-motion revisited. In: Computer Vision and Pattern Recognition (CVPR) (2016)Google Scholar
  37. 37.
    Shariati, A., Holz, C., Sinha, S.: Towards privacy-preserving ego-motion estimation using an extremely low-resolution camera. IEEE Rob. Autom. Lett. (RAL) 5, 1223–1230 (2020)CrossRefGoogle Scholar
  38. 38.
    Shashank, J., Kowshik, P., Srinathan, K., Jawahar, C.: Private content based image retrieval. In: Computer Vision and Pattern Recognition (CVPR) (2008)Google Scholar
  39. 39.
    Shashua, A., Wolf, L.: On the structure and properties of the quadrifocal tensor. In: Vernon, D. (ed.) ECCV 2000, Part I. LNCS, vol. 1842, pp. 710–724. Springer, Heidelberg (2000).  https://doi.org/10.1007/3-540-45054-8_46CrossRefGoogle Scholar
  40. 40.
    Six.D AI: http://6d.ai/ (2018)
  41. 41.
    Speciale, P., Schönberger, J.L., Kang, S.B., Sinha, S.N., Pollefeys, M.: Privacy preserving image-based localization. In: Computer Vision and Pattern Recognition (CVPR) (2019)Google Scholar
  42. 42.
    Speciale, P., Schönberger, J.L., Sinha, S.N., Pollefeys, M.: Privacy preserving image queries for camera localization. In: International Conference on Computer Vision (ICCV) (2019)Google Scholar
  43. 43.
    Strecha, C., von Hansen, W., Gool, L.V., Fua, P., Thoennessen, U.: On benchmarking camera calibration and multi-view stereo for high resolution imagery. In: Computer Vision and Pattern Recognition (CVPR) (2008)Google Scholar
  44. 44.
    Sweeney, L.: k-anonymity: a model for protecting privacy. Int. J. Uncertainty, Fuzziness Knowl. Based Syst. 10, 557–570 (2002)MathSciNetCrossRefGoogle Scholar
  45. 45.
    Upmanyu, M., Namboodiri, A.M., Srinathan, K., Jawahar, C.: Efficient privacy preserving video surveillance. In: International Conference on Computer Vision (ICCV) (2009)Google Scholar
  46. 46.
    Upmanyu, M., Namboodiri, A.M., Srinathan, K., Jawahar, C.: Blind authentication: a secure crypto-biometric verification protocol. IEEE Trans. Inf. Forensics Secur. 5, 255–268 (2010)CrossRefGoogle Scholar
  47. 47.
    Vinje, J.E.: Privacy Manifesto for AR Cloud Solutions (2018). https://medium.com/openarcloud/privacy-manifesto-for-ar-cloud-solutions-9507543f50b6
  48. 48.
    Vondrick, C., Khosla, A., Malisiewicz, T., Torralba, A.: HOGgles: Visualizing object detection features. In: Computer Vision and Pattern Recognition (CVPR) (2013)Google Scholar
  49. 49.
    VPS: Google Visual Positioning System (2018). https://www.engadget.com/2018/05/08/g/
  50. 50.
    Wang, Z., Vineet, V., Pittaluga, F., Sinha, S., Cossairt, O., Kang, S.B.: Privacy-preserving action recognition using coded aperture videos. In: Computer Vision and Pattern Recognition (CVPR) Workshops (2019)Google Scholar
  51. 51.
    Weinzaepfel, P., Jégou, H., Pérez, P.: Reconstructing an image from its local descriptors. In: Computer Vision and Pattern Recognition (CVPR) (2011)Google Scholar
  52. 52.
    Wilson, K., Snavely, N.: Robust global translations with 1DSfM. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014, Part III. LNCS, vol. 8691, pp. 61–75. Springer, Cham (2014).  https://doi.org/10.1007/978-3-319-10578-9_5CrossRefGoogle Scholar
  53. 53.
    Wu, Z., Wang, Z., Wang, Z., Jin, H.: Towards privacy-preserving visual recognition via adversarial training: a pilot study. In: Ferrari, V., Hebert, M., Sminchisescu, C., Weiss, Y. (eds.) ECCV 2018, Part XVI. LNCS, vol. 11220, pp. 627–645. Springer, Cham (2018).  https://doi.org/10.1007/978-3-030-01270-0_37CrossRefGoogle Scholar
  54. 54.
    Yonetani, R., Boddeti, V.N., Kitani, K.M., Sato, Y.: Privacy-preserving visual learning using doubly permuted homomorphic encryption. In: International Conference on Computer Vision (ICCV) (2017)Google Scholar
  55. 55.
    Yosinski, J., Clune, J., Nguyen, A., Fuchs, T., Lipson, H.: Understanding neural networks through deep visualization. In: ICML Workshop on Deep Learning (2015)Google Scholar
  56. 56.
    Zeiler, M.D., Fergus, R.: Visualizing and understanding convolutional networks. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014, Part I. LNCS, vol. 8689, pp. 818–833. Springer, Cham (2014).  https://doi.org/10.1007/978-3-319-10590-1_53CrossRefGoogle Scholar
  57. 57.
    Zhao, J., Frumkin, N., Konrad, J., Ishwar, P.: Privacy-preserving indoor localization via active scene illumination. In: Computer Vision and Pattern Recognition (CVPR) Workshops (2018)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2020

Authors and Affiliations

  1. 1.Department of Computer ScienceETH ZurichZurichSwitzerland
  2. 2.Microsoft MR & AIZurichSwitzerland

Personalised recommendations