Skip to main content
SpringerLink
Log in

Gait transformation network for gait de-identification with pose preservation

  • Original Paper
  • Published:
Signal, Image and Video Processing Aims and scope Submit manuscript

Abstract

Gait and face are the two major biometric features that need to be de-identified to preserve the privacy of an individual and conceal his/her identity. Unlike face de-identification, no research has been conducted on gait de-identification to date. A few existing body/silhouette de-identification approaches use blurring and other primitive image processing techniques that are not robust to varying input environments and tend to remove non-biometric features like appearance and activity. In this paper, we propose a plausible deep learning-based solution to the gait de-identification problem. First, a set of key walking poses is determined from a large gallery set. Next, given an input sequence, a graph-based path search algorithm is employed to classify each frame of the sequence into the appropriate key pose. Next, a random frame with matched key pose chosen from the subset of the gallery sequences is considered the target frame. The dense pose features of the input and target frames are then fused using our proposed gait transformation network (GTNet), which is trained using a combination of perceptual loss, L1 loss, and adversarial loss. Training and testing of the model have been conducted using the RGB sequences present in the TUM-GAID and CASIA-B gait data. Promising de-identification results are obtained both qualitatively and quantitatively.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

Availability of data and materials

Pretrained model will be made available on paper acceptance.

References

  1. Agarwal, A., Chattopadhyay, P., Wang, L.: Privacy preservation through facial de-identification with simultaneous emotion preservation. Signal Image Video Process. 15(5), 951–958 (2021)

    Article  Google Scholar 

  2. Gafni, O., Wolf, L., Taigman, Y.: Live face de-identification in video. In Proc. of the IEEE/CVF Intl. Conf. on Computer Vision, pp. 9378–9387. (2019)

  3. Ma, L., Jia, X., Sun, Q., Schiele, B., Tuytelaars, T., Van Gool, L.: Pose guided person image generation. In: Proc. of the Advances in Neural Information Processing Systems, 30. (2017)

  4. Ma, L., Sun, Q., Georgoulis, S., Van Gool, L., Schiele, B., Fritz, M.: Disentangled person image generation. In Proc. of the IEEE Conf. on Computer Vision and Pattern Recognition, pp. 99–108. (2018)

  5. Agrawal, P., Narayanan, P.J.: Person de-identification in videos. IEEE Trans. Circuits Syst. Video Technol. 21(3), 299–310 (2011)

    Article  Google Scholar 

  6. Boyle, M., Edwards, C., Greenberg, S.: The effects of filtered video on awareness and privacy. In: Proc. of the 2000 ACM Conf. on Computer Supported Cooperative Work, pp. 1–10. (2000)

  7. Park, S., Trivedi, M. M.: A track-based human movement analysis and privacy protection system adaptive to environmental contexts. In: Proc. of the IEEE Conf. on Advanced Video and Signal Based Surveillance, pp. 171–176. (2005)

  8. Brkic, K., Sikiric, I., Hrkac, T., Kalafatic, Z.: I know that person: Generative full body and face de-identification of people in images. In: Proc. of the IEEE Conf. on Computer Vision and Pattern Recognition Workshops, pp. 1319–1328. (2017)

  9. Gatys, L., Ecker, A.S., Bethge, M.: Texture synthesis using convolutional neural networks. In: Proc. of the Advances in Neural Information Processing Systems, 28. (2015)

  10. Gatys, L.A., Ecker, A.S., Bethge, M.: A neural algorithm of artistic style. arXiv preprint arXiv:1508.06576. (2015)

  11. Ulyanov, D., Lebedev, V., Vedaldi, A., Lempitsky, V.S.: Texture networks: feed-forward synthesis of textures and stylized images. In: Proc. of the Intl. Conf. on Machine Learning, pp. 1349–1357. (2016)

  12. Gross, R., Sweeney, L., De la Torre, F., Baker, S.: Model-based face de-identification. In: Proc. of the Conf. on Computer Vision and Pattern Recognition Workshop, pp. 161–168. (2006)

  13. Neustaedter, C., Greenberg, S., Boyle, M.: Blur filtration fails to preserve privacy for home-based video conferencing. ACM Trans. Comput. Hum. Interact. 13(1), 1–36 (2006)

    Article  Google Scholar 

  14. Oh, S.J., Benenson, R., Fritz, M., Schiele, B.: Faceless person recognition: privacy implications in social media. In: Proc. of the European Conf. on Computer Vision, pp. 19–35. (2016)

  15. Aggarwal, A., Rathore, R., Chattopadhyay, P., Wang, L.: EPD-net: A GAN-based architecture for face de-identification from images. In: Proc. of the IEEE Intl. IOT, Electronics and Mechatronics Conf., pp. 1–7. (2020)

  16. Newton, E.M., Sweeney, L., Malin, B.: Preserving privacy by de-identifying face images. IEEE Trans. Knowl. Data Eng. 17(2), 232–243 (2005)

    Article  Google Scholar 

  17. Gross, R., Airoldi, E., Malin, B., Sweeney, L.: Integrating utility into face de-identification. In: Proc. of the Intl. Workshop on Privacy Enhancing Technologies, pp. 227–242. (2005)

  18. Goodfellow, I.J., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., Bengio, Y.: Generative adversarial nets. In: Proc. of the Advances in Neural Information Processing Systems, 27. (2014)

  19. Wen, Y., Bo, L., Cao, J., Xie, R., Song, L., Li, Z.: IdentityMask: deep motion flow guided reversible face video de-identification. IEEE Trans. Circuits Syst. Video Technol. (2022). https://doi.org/10.1109/TCSVT.2022.3191982

    Article  Google Scholar 

  20. Korshunova, I., Shi, W., Dambre, J., Theis, L.: Fast face-swap using convolutional neural networks. In: Proc. of the IEEE Intl. Conf. on Computer Vision, pp. 3677–3685. (2017)

  21. Dou, S., Jiang, X., Zhao, Q., Li, D., Zhao, C.: Towards privacy-preserving person re-identification via person identify shift. arXiv preprint arXiv:2207.07311. (2022)

  22. Ren, Z., Lee, Y.J., Ryoo, M.S.: Learning to anonymize faces for privacy preserving action detection. In: Proc. of the European Conf. on Computer Vision, pp. 620–636. (2018)

  23. Mirza, M., Osindero, S.: Conditional generative adversarial nets. arXiv preprint arXiv:1411.1784. (2014)

  24. Gong, M., Liu, J., Li, H., Xie, Yu., Tang, Z.: Disentangled representation learning for multiple attributes preserving face deidentification. IEEE Trans. Neural Netw. Learn. Syst. (2020). https://doi.org/10.1109/TNNLS.2020.3027617

    Article  Google Scholar 

  25. Chen, D., Chang, Y., Yan, R., Yang, J.: Tools for protecting the privacy of specific individuals in video. EURASIP J. Adv. Signal Process. 2007, 1–9 (2007)

    Article  MATH  Google Scholar 

  26. Roy, A., Sural, S., Mukherjee, J.: Gait recognition using pose kinematics and pose energy image. Signal Process. 92(3), 780–792 (2012)

    Article  Google Scholar 

  27. Long, J., Shelhamer, E., Darrell, T.: Fully convolutional networks for semantic segmentation. In: Proc. of the IEEE Conf. on Computer Vision and Pattern Recognition, pp. 3431–3440. (2015)

  28. Gupta, S.K., Chattopadhyay, P.: Gait recognition in the presence of co-variate conditions. Neurocomputing 454, 76–87 (2021)

  29. Güler, RA., Neverova, N., Kokkinos, I.: Densepose: dense human pose estimation in the wild. In: Proc. of the IEEE Conf. on Computer Vision and Pattern Recognition, pp. 7297–7306. (2018)

  30. Albahar, B., Jingwan, L., Yang, J., Shu, Z., Shechtman, E., Huang, J.-B.: Pose with style: Detail-preserving pose-guided image synthesis with conditional stylegan. ACM Trans. Graph. 40(6), 1–11 (2021)

  31. Zhu, Z., Huang, T., Shi, B., Yu, M., Wang, B., Bai, X.: Progressive pose attention transfer for person image generation. In: Proc. of the IEEE/CVF Conf. on Computer Vision and Pattern Recognition, pp. 2347–2356. (2019)

  32. SSimonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556. (2014)

  33. Hofmann, M., Geiger, J., Bachmann, S., Schuller, B., Rigoll, G.: The tum gait from audio, image and depth (GAID) database: multimodal recognition of subjects and traits. J. Vis. Commun. Image Represent. 25(1), 195–206 (2014)

    Article  Google Scholar 

  34. Yu, S., Tan, D., Tan, T.: A framework for evaluating the effect of view angle, clothing and carrying condition on gait recognition. In: Proc. of the \(18^{th}\) Intl. Conf. on Pattern Recognition, vol. 4, pp. 441–444. (2006)

  35. Chao, H., He, Y., Zhang, J., Feng, J.: Gaitset: regarding gait as a set for cross-view gait recognition. In: Proc. of the AAAI Conf. on Artificial Intelligence, vol. 33, 01, pp. 8126–8133. (2019)

  36. Hou, S., Cao, C., Liu, X., Huang, Y.: Gait lateral network: Learning discriminative and compact representations for gait recognition. In: Proc. of the European Conf. on Computer Vision, pp. 382–398, (2020)

  37. Fan, C., Peng, Y., Cao, C., Liu, X., Hou, S., Chi, J., Huang, Y., Li, Q., He, Z.: Gaitpart: temporal part-based model for gait recognition. In: Proc. of the IEEE/CVF Conf. on Computer Vision and Pattern Recognition, pp. 14225–14233. (2020)

  38. Lin, B., Zhang, S., Yu, X.: Gait recognition via effective global-local feature representation and local temporal aggregation. In: Proc. of the IEEE/CVF Intl. Conf. on Computer Vision, pp. 14648–14656. (2021)

Download references

Acknowledgements

The authors would like to thank SERB, DST, GoI for supporting this work with a server.

Funding

The server used in the work is purchased from grant CRG/2020/005465 (SERB, DST, GoI).

Author information

Authors and Affiliations

  1. Pattern Recognition Lab, Department of CSE, IIT (BHU), Varanasi, 221005, India

    Agrya Halder & Pratik Chattopadhyay

  2. Department of Electrical Engineering and Computer Science, Washkewicz College of Engineering, Cleveland, OH, 44115, USA

    Sathish Kumar

Authors
  1. Agrya Halder
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Pratik Chattopadhyay
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Sathish Kumar
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

AH has done the concept modeling, implementation, and manuscript writing. PC supervised the entire work and helped AH in formulating the concepts and also thoroughly checked the manuscript. SK has also helped in concept building and proof-reading of the entire paper.

Corresponding author

Correspondence to Pratik Chattopadhyay.

Ethics declarations

Conflict of interest

All authors declare that they have no conflict of interest.

Consent for publication

Not applicable.

Ethics approval and consent to participate

Not applicable.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Halder, A., Chattopadhyay, P. & Kumar, S. Gait transformation network for gait de-identification with pose preservation. SIViP 17, 1753–1761 (2023). https://doi.org/10.1007/s11760-022-02386-x

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11760-022-02386-x

Keywords

Search

Navigation