Skip to main content

Learning to Censor by Noisy Sampling

  • Conference paper
  • First Online:
Computer Vision – ECCV 2022 (ECCV 2022)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 13673))

Included in the following conference series:

  • 2015 Accesses

Abstract

Point clouds are an increasingly ubiquitous input modality and the raw signal can be efficiently processed with recent progress in deep learning. This signal may, often inadvertently, capture sensitive information that can leak semantic and geometric properties of the scene which the data owner does not want to share. The goal of this work is to protect sensitive information when learning from point clouds; by censoring the sensitive information before the point cloud is released for downstream tasks. Specifically, we focus on preserving utility for perception tasks while mitigating attribute leakage attacks. The key motivating insight is to leverage the localized saliency of perception tasks on point clouds to provide good privacy-utility trade-offs. We realise this through a mechanism called Censoring by Noisy Sampling (CBNS), which is composed of two modules: i) Invariant Sampling: a differentiable point-cloud sampler which learns to remove points invariant to utility and ii) Noise Distorter: which learns to distort sampled points to decouple the sensitive information from utility, and mitigate privacy leakage. We validate the effectiveness of CBNS through extensive comparisons with state-of-the-art baselines and sensitivity analyses of key design choices. Results show that CBNS achieves superior privacy-utility trade-offs on multiple datasets. The code for our work can be found https://github.com/java-abhinav07/CBNS-ECCV2022.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Subscribe and save

Springer+ Basic
EUR 32.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or Ebook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 89.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 119.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

Notes

  1. 1.

    https://leginfo.legislature.ca.gov/faces/billTextClient.xhtml?bill_id=201720180AB375.

References

  1. Qi, C.R., Su, H., Mo, K., Guibas, L.J.: PointNet: deep learning on point sets for 3d classification and segmentation (2017)

    Google Scholar 

  2. Qi, C.R., Yi, L., Su, H., Guibas, L.J.: PointNet++: deep hierarchical feature learning on point sets in a metric space. In: 30th Proceedings of the Conference on Advances in Neural Information Processing Systems (2017)

    Google Scholar 

  3. Wang, Y., et al.: Dynamic graph CNN for learning on point clouds. ACM Trans. Graph. 38(5), 1–12 (2019)

    Article  Google Scholar 

  4. Li, Y., Bu, R., Sun, M., Wu, W., Di, X., Chen, B.: PointCNN: convolution on x-transformed points. In: 31st Proceedings of the Conference on Advances in Neural Information Processing Systems (2018)

    Google Scholar 

  5. Wu, W., Qi, Z., Fuxin, L.: PointConv : deep convolutional networks on 3D point clouds. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition., pp. 9621–9630 (2019)

    Google Scholar 

  6. Zhao, H., Jiang, L., Jia, J., Torr, P.H., Koltun, V.: Point transformer. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 16259–16268 (2021)

    Google Scholar 

  7. Xu, C., et al.: SqueezeSegV3: spatially-adaptive convolution for efficient point-cloud segmentation. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, J.-M. (eds.) ECCV 2020. LNCS, vol. 12373, pp. 1–19. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-58604-1_1

    Chapter  Google Scholar 

  8. Hu, Q., et al.: Randla-Net: efficient semantic segmentation of large-scale point clouds. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 11108–11117 (2020)

    Google Scholar 

  9. Zhang, Y., Zhou, Z., David, P., Yue, X., Xi, Z., Gong, B., Foroosh, H.: PolarNet: an improved grid representation for online lidar point clouds semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 9601–9610 (2020)

    Google Scholar 

  10. Chen, C., Qian, S., Fang, Q., Xu, C.: HAPGN: hierarchical attentive pooling graph network for point cloud segmentation. IEEE Trans. Multim. 23, 2335–2346 (2020)

    Article  Google Scholar 

  11. Qi, C.R., Liu, W., Wu, C., Su, H., Guibas, L.J.: Frustum pointnets for 3D object detection from RGB-D data. In: Proceedings of the IEEE conference on Computer Vision and Pattern Recognition, pp. 918–927 (2018)

    Google Scholar 

  12. Qi, C.R., Litany, O., He, K., Guibas, L.J.: Deep Hough voting for 3d object detection in point clouds. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 9277–9286 (2019)

    Google Scholar 

  13. Martin, S., et al.: Complex-yolo: real-time 3D objectdetection on point clouds. In: Computer Vision and Pattern Recognition. (2018)

    Google Scholar 

  14. Shi, S., Wang, X., Li, H.: PointrCnn: 3D object proposal generation and detection from point cloud. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 770–779 (2019)

    Google Scholar 

  15. Sarode, V., Li, X., Goforth, H., Aoki, Y., Srivatsan, R.A., Lucey, S., Choset, H.: PcrNet: point cloud registration network using pointnet encoding. arXiv preprint arXiv:1908.07906 (2019)

  16. Aoki, Y., Goforth, H., Srivatsan, R.A., Lucey, S.: PointNetLK: robust & efficient point cloud registration using pointNet. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 7163–7172 (2019)

    Google Scholar 

  17. Song, C., Shmatikov, V.: Overlearning reveals sensitive attributes. arXiv preprint arXiv:1905.11742 (2019)

  18. Atchinson, B.K., Fox, D.M.: From the field: the politics of the health insurance portability and accountability act. Health Aff. 16(3), 146–150 (1997)

    Article  Google Scholar 

  19. Arora, S., Liang, Y., Ma, T.: Why are deep nets reversible: a simple theory, with implications for training. CoRR abs/1511.05653 (2015)

    Google Scholar 

  20. Speciale, P., Kang, S.B., Pollefeys, M., Schönberger, J., Sinha, S.: Privacy preserving image-based localization. In: 2019 Conference on Computer Vision and Pattern Recognition (CVPR), IEEE, June 2019

    Google Scholar 

  21. Speciale, P., Schonberger, J.L., Sinha, S.N., Pollefeys, M.: Privacy preserving image queries for camera localization. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 1486–1496 (2019)

    Google Scholar 

  22. Shibuya, M., Sumikura, S., Sakurada, K.: Privacy preserving visual SLAM. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, J.-M. (eds.) ECCV 2020. LNCS, vol. 12367, pp. 102–118. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-58542-6_7

    Chapter  Google Scholar 

  23. Geppert, M., Larsson, V., Speciale, P., Schönberger, J.L., Pollefeys, M.: Privacy preserving structure-from-motion. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, J.-M. (eds.) ECCV 2020. LNCS, vol. 12346, pp. 333–350. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-58452-8_20

    Chapter  Google Scholar 

  24. Jia, J., Gong, N.Z.: \(\{\)AttriGuard\(\}\): a practical defense against attribute inference attacks via adversarial machine learning. In: 27th USENIX Security Symposium (USENIX Security 18), pp. 513–529 (2018)

    Google Scholar 

  25. Chelani, K., Kahl, F., Sattler, T.: How privacy-preserving are line clouds? recovering scene details from 3D lines (2021)

    Google Scholar 

  26. Dwork, C., McSherry, F., Nissim, K., Smith, A.: Calibrating noise to sensitivity in private data analysis. In: Halevi, S., Rabin, T. (eds.) TCC 2006. LNCS, vol. 3876, pp. 265–284. Springer, Heidelberg (2006). https://doi.org/10.1007/11681878_14

    Chapter  Google Scholar 

  27. Hamm, J.: Minimax filter: learning to preserve privacy from inference attacks. J. Mach. Learn. Res. 18(129), 1–31 (2017)

    MathSciNet  MATH  Google Scholar 

  28. Singh, A., et al.: DISCO: dynamic and invariant sensitive channel obfuscation for deep neural networks. In: 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2021)

    Google Scholar 

  29. Wu, Z., Song, S., Khosla, A., Yu, F., Zhang, L., Tang, X., Xiao, J.: 3D shapeNets: a deep representation for volumetric shapes. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1912–1920 (2015)

    Google Scholar 

  30. Roy, P.C., Boddeti, V.N.: Mitigating information leakage in image representations: a maximum entropy approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 2586–2594

    Google Scholar 

  31. Lang, I., Manor, A., Avidan, S.: SampleNet: differentiable point cloud sampling. In: 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2020)

    Google Scholar 

  32. Osia, S.A., et al.: A hybrid deep learning architecture for privacy-preserving mobile analytics. IEEE Internet Things J. 7(5), 4505–4518 (2020)

    Article  Google Scholar 

  33. Mireshghallah, F., Taram, M., Ramrakhyani, P., Jalali, A., Tullsen, D., Esmaeilzadeh, H.: Shredder: Learning noise distributions to protect inference privacy. In: Proceedings of the Twenty-Fifth International Conference on Architectural Support for Programming Languages and Operating Systems, pp. 3–18 (2020)

    Google Scholar 

  34. Yang, H., et al.: FaceScape: a large-scale high quality 3d face dataset and detailed riggable 3D face prediction. In: IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). June 2020

    Google Scholar 

  35. Ravi, N., et al.: Accelerating 3d deep learning with pytorch3d. arXiv:2007.08501 (2020)

  36. Fan, L.: Image pixelization with differential privacy. In: Kerschbaum, F., Paraboschi, S. (eds.) DBSec 2018. LNCS, vol. 10980, pp. 148–162. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-95729-6_10

    Chapter  Google Scholar 

  37. Ishibuchi, H., Imada, R., Setoguchi, Y., Nojima, Y.: How to specify a reference point in hypervolume calculation for fair performance comparison. Evol. Comput. 26(3), 411–440 (2018)

    Article  Google Scholar 

  38. Zheng, T., Chen, C., Yuan, J., Li, B., Ren, K.: PointCloud saliency maps. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 1598–1606 (2019)

    Google Scholar 

  39. Samarati, P., Sweeney, L.: Protecting privacy when disclosing information: k-anonymity and its enforcement through generalization and suppression. J. Am. Med. Inform. Assoc. 15(5), 627–637 (1998)

    Google Scholar 

  40. Wang, T., Zhang, X., Feng, J., Yang, X.: A comprehensive survey on local differential privacy toward data statistics and analysis. Sensors 20(24), 7030 (2020)

    Article  Google Scholar 

  41. Xiao, T., Tsai, Y.H., Sohn, K., Chandraker, M., Yang, M.H.: Adversarial learning of privacy-preserving and task-oriented representations. In: Proceedings of the AAAI Conference on Artificial Intelligence., vol.34, pp. 12434–12441 (2020)

    Google Scholar 

  42. Liu, Z., Wu, Z., Gan, C., Zhu, L., Han, S.: DataMix: efficient privacy-preserving edge-cloud inference. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, J.-M. (eds.) ECCV 2020. LNCS, vol. 12356, pp. 578–595. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-58621-8_34

    Chapter  Google Scholar 

  43. McMahan, H.B., Moore, E., Ramage, D., y Arcas, B.A.: Federated learning of deep networks using model averaging. arXiv preprint arXiv:1602.05629 (2016)

  44. Gupta, O., Raskar, R.: Distributed learning of deep neural network over multiple agents. J. Netw. Comput. Appl. 116, 1–8 (2018)

    Article  Google Scholar 

  45. Du, J., Li, S., Feng, M., Chen, S.: Dynamic differential-privacy preserving sgd. arXiv preprint arXiv:2111.00173 (2021)

  46. Ho, S., Qu, Y., Gu, B., Gao, L., Li, J., Xiang, Y.: DP-GAN: differentially private consecutive data publishing using generative adversarial nets. J. Netw. Comput. Appl. 185, 103066 (2021)

    Article  Google Scholar 

  47. Jordon, J., Yoon, J., Van Der Schaar, M.: Pate-GAN: generating synthetic data with differential privacy guarantees. In: International Conference on Learning Representations. (2018)

    Google Scholar 

  48. Liu, Z., Hu, H., Cao, Y., Zhang, Z., Tong, X.: A Closer Look at Local Aggregation Operators in Point Cloud Analysis. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, J.-M. (eds.) ECCV 2020. LNCS, vol. 12368, pp. 326–342. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-58592-1_20

    Chapter  Google Scholar 

  49. Yan, X., Zheng, C., Li, Z., Wang, S., Cui, S.: PointASNL: robust point clouds processing using nonlocal neural networks with adaptive sampling. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 5589–5598 (2020)

    Google Scholar 

  50. Bytyqi, Q., Wolpert, N., Schömer, E.: Local-area-learning network: Meaningful local areas for efficient point cloud analysis. arXiv preprint arXiv:2006.07226 (2020)

  51. Xu, Q., Sun, X., Wu, C.Y., Wang, P., Neumann, U.: GRID-GCN for fast and scalable point cloud learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 5661–5670 (2020)

    Google Scholar 

  52. Xiang, T., Zhang, C., Song, Y., Yu, J., Cai, W.: Walk in the cloud: learning curves for point clouds shape analysis. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 915–924

    Google Scholar 

  53. Lin, C., Li, C., Liu, Y., Chen, N., Choi, Y.K., Wang, W.: Point2skeleton: Learning skeletal representations from point clouds. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4277–4286 (2021)

    Google Scholar 

  54. Lang, I., Kotlicki, U., Avidan, S.: Geometric adversarial attacks and defenses on 3D point cloud. In: 2021 International Conference on 3D Vision (3DV) (2021)

    Google Scholar 

  55. Yang, J., Zhang, Q., Fang, R., Ni, B., Liu, J., Tian, Q.: Adversarial attack and defense on point sets (2021)

    Google Scholar 

  56. Liu, D., Yu, R., Su, H.: Extending adversarial attacks and defenses to deep 3D point cloud classifiers (2019)

    Google Scholar 

  57. Lee, K., Chen, Z., Yan, X., Urtasun, R., Yumer, E.: ShapeAdv: generating shape-aware adversarial 3d point clouds. arXiv preprint arXiv:2005.11626 (2020)

  58. Dovrat, O., Lang, I., Avidan, S.: Learning to sample. In:: 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2755–2764 (2019)

    Google Scholar 

  59. Gupta, A., Ligett, K., McSherry, F., Roth, A., Talwar, K.: Differentially private combinatorial optimization. In: Proceedings of the Twenty-first Annual ACM-SIAM Symposium on Discrete Algorithms, SIAM, pp. 1106–1125 (2010)

    Google Scholar 

  60. Feldman, D., Fiat, A., Kaplan, H., Nissim, K.: Private coresets. In: Proceedings of the Forty-First Annual ACM Symposium on Theory of computing, pp. 361–370 (2009)

    Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Ayush Chopra .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and permissions

Copyright information

© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Chopra, A., Java, A., Singh, A., Sharma, V., Raskar, R. (2022). Learning to Censor by Noisy Sampling. In: Avidan, S., Brostow, G., Cissé, M., Farinella, G.M., Hassner, T. (eds) Computer Vision – ECCV 2022. ECCV 2022. Lecture Notes in Computer Science, vol 13673. Springer, Cham. https://doi.org/10.1007/978-3-031-19778-9_22

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-19778-9_22

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-19777-2

  • Online ISBN: 978-3-031-19778-9

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics