Oblivious Image Matching

  • Shai Avidan
  • Ariel Elbaz
  • Tal Malkin
  • Ryan Moriarty


Video surveillance is an intrusive operation that violates privacy. It is therefore desirable to devise surveillance protocols that minimize or even eliminate privacy intrusion. A principled way of doing so is to resort to Secure Multi-Party methods, that are provably secure, and adapt them to various vision algorithms. In this chapter, we describe an Oblivious Image Matching protocol which is a secure protocol for image matching. Image matching is a generalization of detection and recognition tasks since detection can be viewed as matching a particular image to a given object class (i.e., does this image contain a face?) while recognition can be viewed as matching an image of a particular instance of a class to another image of the same instance (i.e., does this image contain a particular car?). And instead of applying the Oblivious Image Matching to the entire image one can apply it to various sub-images, thus solving the localization problem (i.e., where is the gun in the image?). A leading approach to object detection and recognition is the bag-offeatures approach, where each object is reduced to a set of features and matching objects is reduced to matching their corresponding sets of features. Oblivious Image Matching uses a secure fuzzy match of string and sets as its building block. In the proposed protocol, two parties, Alice and Bob, wish to match their images, without leaking additional information. We use a novel cryptographic protocol for fuzzy matching and adopt it to the bag-of-features approach. Fuzzy matching compares two sets (or strings) and declares them to match if a certain percentage of their elements match. To apply fuzzy matching to images, we represent images as a set of visual words that can be fed to the secure fuzzy matching protocol. The fusion of a novel cryptographic protocol and recent advances in computer vision results in a secure and efficient protocol for image matching. Experiments on real images are presented.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. [AB06]
    S. Avidan and M. Butman. Blind vision. In European Conference on Computer Vision, pages 1–13, 2006.Google Scholar
  2. [AEMM07]
    S. Avidan, A. Elbaz, T. Malkin, and R. Moriarty. Oblivious image matching. Technical Report cucs-030-07, Columbia University, 2007.Google Scholar
  3. [BNJ03]
    D. Blei, A. Ng, and M. Jordan. Latent dirchilet allocation. Journal of Machine Learning Research, 3:993–1022, 2003.MATHCrossRefGoogle Scholar
  4. [CGKS95]
    B. Chor, O. Goldreich, E. Kushilevitz, and M. Sudan. Private information retrieval. In IEEE Symposium on Foundations of Computer Science, pages 41–50, 1995.Google Scholar
  5. [CMS99]
    C. Cachin, S. Micali, and M. Stadler. Computationally private information retrieval with polylogarithmic communication. Lecture Notes in Computer Science, 1592:402–414, 1999.CrossRefGoogle Scholar
  6. [FFFP04]
    L. Fei-Fei, R. Fergus, and P. Perona. Learning generative visual models from few training examples. In Workshop on Generative-Model Based Vision, IEEE Proc. CVPR, 2004.Google Scholar
  7. [FNP04]
    M. Freedman, K. Nissim, and B. Pinkas. Efficient private matching and set intersection. In Advances in Cryptology – (EUROCRYPT 2004), volume 3027, pages 1–19, Springer-Verlag, 2004.Google Scholar
  8. [Gol04]
    O. Goldreich. Foundations of Cryptography. Cambridge University Press, 2004.Google Scholar
  9. [Hof99]
    T. Hofmann. Probabilistic latent semantic indexing. In SIGIR, pages 50–57, 1999.Google Scholar
  10. [Low04]
    D. G. Lowe. Distinctive image features from scale-invariant keypoints. International Journal of Computer Vision, 60(2):91–110, 2004.CrossRefGoogle Scholar
  11. [MS04]
    K. Mikolajczyk and C. Schmid. Scale and affine invariant interest point detectors. International Journal of Computer Vision, 60(1):63–86, 2004.CrossRefGoogle Scholar
  12. [MS05]
    K. Mikolajczyk and C. Schmid. A performance evaluation of local descriptors. IEEE Transactions on Pattern Recognition and Machine Intelligence, 27(10):1615–1630, 2005.CrossRefGoogle Scholar
  13. [NS06]
    D. Nistér and H. Stewénius. Scalable recognition with a vocabulary tree. In IEEE Conference on Computer Vision and Pattern Recognition, pages 2161–2168, 2006.Google Scholar
  14. [SZ03]
    J. Sivic and A. Zisserman. Video google: A text retrieval approach to object matching in videos. International Conference on Computer Vision, pages 1470–1477, 2003.Google Scholar
  15. [SZ08]
    J. Sivic and A. Zisserman. Efficient visual search for objects in videos. Proceedings of the IEEE, 96(4), 2008.Google Scholar
  16. [Yao82]
    A. C. Yao. Protocols for secure computations. In Proc. 23rd IEEE Symposium on Foundations of Comp. Science, pages 160–164, Chicago, 1982.Google Scholar

Copyright information

© Springer Science+Business Media, LLC 2009

Authors and Affiliations

  • Shai Avidan
    • 1
  • Ariel Elbaz
    • 2
  • Tal Malkin
    • 3
  • Ryan Moriarty
    • 4
  1. 1.Adobe Systems IncNewtonUSA
  2. 2.Columbia UniversityNew YorkUSA
  3. 3.Columbia UniversityNew YorkUSA
  4. 4.University of CaliforniaUSA

Personalised recommendations