Skip to main content
SpringerLink
Log in
Book cover

Digital Hampi: Preserving Indian Cultural Heritage pp 71–90Cite as

Acquisition, Representation, Processing and Display of Digital Heritage Sites

Abstract

This chapter presents salient aspects of the research undertaken on the project Acquisition, Representation, Processing and Display of Digital Heritage Sites. The main objective of the project was to create algorithms and techniques to acquire a three-dimensional digital replica of complex structures spread over a large area. The techniques developed are applied to Hampi, a world heritage site. In addition to acquiring the geometry and surface properties, we also research efficient representation and visualisation of this data and provide tools and methods for users to experience the captured models, to virtual walk-through and explore the digital recreations. For the acquisition, we rely on multimodal input using technologies like laser scanners, colour cameras and depth sensors. We align and fuse geometric constructions from different modalities through a step of registration. We have extended structure from motion (SfM), a state-of-the-art approach for multi-view 3D reconstruction from images and developed techniques for large-scale (relatively) sparse geometric constructions and simultaneously dense reconstructions of smaller parts. We also provide ability to generate high-resolution point cloud from the point cloud obtained from depth camera Kinect by using additional high definition cameras. We also explore efficient visualisation of large models with augmented reality and user experience authoring. Hampi has been chosen as a test bench for developing our techniques. Within Hampi, we concentrate on Vittala Temple Complex, and demonstrate our techniques on it. The project has greatly benefitted from the collaboration from other partner institutes especially BVBCET, NID, NIAS, IIT Bombay, IIT Madras and IIIT Hyderabad and IISc Bangalore.

Keywords

  • Kinect Point Cloud
  • Structure From Motion (SfM)
  • Single View Reconstruction
  • Pairwise Epipolar Geometry
  • Image Connectivity

These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

This is a preview of subscription content, access via your institution.

References

  1. Agarwal S, Snavely N, Seitz S, Szeliski R (2010) Bundle adjustment in the large. In: Proceedings of the European conference on computer vision, pp 29–42

    Google Scholar 

  2. Agarwal S, Snavely N, Simon I, Seitz S, Szeliski R (2009) Building rome in a day. In: Proceedings of the international conference on computer vision, pp 72–79

    Google Scholar 

  3. Arora N, Kalra P (2011) Interactive image restoration using inpainting and denoising. In: NCVPRIPG 2011—Proceedings of national conference on computer vision, pattern recognition, image processing and graphics, Hubli, 15–17 December, 2011, pp 219–222

    Google Scholar 

  4. Arora N, Kumar A, Kalra P (2013) Digital restoration of old paintings. In: WSCG 2013—Proceedings of conference on computer graphics, visualization and computer vision plzen, Czech Republic 24–27 June, 2013, p F89

    Google Scholar 

  5. Bhowmick B, Patra S, Chaterjee A, Govindu VM, Banerjee S (2014) Divide and conquer: efficient large scalre structure from motion using graph partitioning. Proc ACCV 2014:273–287

    Google Scholar 

  6. Boykov Y, Kolmogorov V (2004) An experimental comparison of min-cut/max-flow algorithms for energy minimization in vision. In: IEEE transactions on PAMI, vol 26, no 9, September 2004, pp 1124–1137

    Google Scholar 

  7. Bugalia N (2016) Immersive environment system for an efficient human computer interaction. MSR thesis, Amar Nath and Shashi Khosla School of Information Technology, IIT Delhi

    Google Scholar 

  8. Bugalia N, Kumar S, Kalra P, Choudhary S (2016) Mixed reality based interaction system for digital heritage. In: Proceedings of the 15th ACM SIGGRAPH conference on virtual-reality continuum and its applications in industry, VRCAI 2016, Zhuhai, China, December 3–4, 2016, pp 31–37

    Google Scholar 

  9. Crandall DJ, Owens A, Snavely N, Huttenlocher DP (2011) Discrete-continuous optimization for large-scale structure from motion. In: Proceedings of IEEE conference on computer vision and pattern recognition, pp 3001–3008

    Google Scholar 

  10. Crandall DJ, Owens A, Snavely N, Huttenlocher DP (2013) SfM with MRFs: discrete-continuous optimization for large-scale reconstruction. IEEE Trans Pattern Anal Mach Intell 35(12):2841–2853

    CrossRef  Google Scholar 

  11. Criminisi A, Reid I, Zisserman A (2000) Single view metrology. Int J Comput Vis 40(2):123–148

    Google Scholar 

  12. Diebel J, Thrun S (2006) An application of markov random fields to range sensing. In: Advances in neural information processing systems, pp 291–298

    Google Scholar 

  13. Farenzena M, Fusiello A, Gherardi R (2009) Structure-and-motion pipeline on a hierarchical cluster tree. In: Proceedings of IEEE international conference on computer vision workshop on 3-D digital imaging and modeling, pp 1489–1496

    Google Scholar 

  14. Frahm J, Georgel P, Gallup D, Johnson T, Raguram R, Wu C, Jen Y, Dunn E, Clipp B, Lazebnik S, Pollefeys M (2010) Building Rome on a cloudless day. In: Proceedings of the European conference on computer vision: Part IV, pp 368–381

    Google Scholar 

  15. Furukawa Y (2016) Clustering views for multi-view stereo. http://www.di.ens.fr/cmvs. Accessed 26 July 26 2016

  16. Havlena M, Torii A, Pajdla T (2010) Efficient structure from motion by graph optimization. In: Proceedings of the European conference on computer vision, Lecture notes in computer science, vol 6312, pp 100–113

    Google Scholar 

  17. Hoiem D, Efros AA, Hebert M (2005) Automatic photo pop-up. Proc SIGGRAPH 2005:577–584

    Google Scholar 

  18. Horry Y, Anjyo KI, Arai K (1997) Tour into the picture: using a spidery mesh interface to make animation from a single image. Proc SIGGRAPH 97:225–232

    Google Scholar 

  19. Jiang N, Cui Z, Tan P (2013) A global linear method for camera pose registration. In: Proceedings of IEEE international conference on computer vision, pp 481–488

    Google Scholar 

  20. Khurana D, Sankhla S, Shukla A, Varshney R, Kalra P, Banerjee S (2012) A grammar-based gui for single view reconstruction. In: Proceedings of ICVGIP 2012, p 14

    Google Scholar 

  21. Kushal A, Chanda G, Srivastava K, Gupta M, Sanyal S, Sriram T, Kalra P, Banerjee S (2003) Multilevel modelling and rendering of architectural scenes. In: Proceedings of EUROGRAPHICS 2003 short presentations

    Google Scholar 

  22. Kutulakos K, Seitz S (1999) A theory of shape by space carving. In: 7th IEEE international conference on computer vision (ICCV-99), vol I, pp 307–314

    Google Scholar 

  23. Laycock RG, Drinkwater D, Day AM (2008) Exploring cultural heritage sites through space and time. J Comput Cult Heritage 1(2):1–15

    CrossRef  Google Scholar 

  24. Lourakis M, Argyros A (2007) Enforcing scene constraints in single view reconstruction. In: Proceedings of EUROGRAPHICS 2007 short papers, pp 45–48

    Google Scholar 

  25. Lowe D (2004) Distinctive image features from scale-invariant keypoints. Int J Comput Vis 60(2):91–110

    CrossRef  Google Scholar 

  26. Magnenat-Thalmann N, Foni AE, Cadi-Yazli N (2006) Real-time animation of ancient roman sites. In: Proceedings of 4th international conference on computer graphics and interactive techniques, GRAPHITE, pp 19–30

    Google Scholar 

  27. Moulon P, Monasse P, Marlet R (2013) Global fusion of relative motions for robust, accurate and scalable structure from motion. In: Proceedings of IEEE international conference on computer vision, pp 3248–3255

    Google Scholar 

  28. Muller P, Zeng G, Wonka P, Gool LV (2007) Image-based procedural modeling of facades. Proc SIGGRAPH 2007:181–184

    Google Scholar 

  29. Nister D, Stewenius H (2006) Scalable recognition with a vocabulary tree. Proc IEEE Conf Comput Vis Pattern Recogn 2:2161–2168

    Google Scholar 

  30. Papagiannakis G, Schertenleib S, O’Kennedy B, Arevalo-Pozat M, Magnenat-Thalmann N, Thalmann D (2005) Mixing virtual and real scenes in the site of ancient pompeii. Comput Anim Virtual Worlds 16(1):11–24

    CrossRef  Google Scholar 

  31. Patra S, Bhowmick B, Kalra P, Banerjee S (2012) Kinect. In: VISIAPP 2012

    Google Scholar 

  32. Petridis P, White M, Mourkousis N, Liarokapis F, Sifiniotis M, Gatzidis C (2009) Exploring and interacting with virtual museums. J Comput Cult Heritage 2(1):1–20

    Google Scholar 

  33. Raghuram R, Wu C, Frahm J, Lazebnik S (2011) Modeling and recognition of landmark image collections using iconic scene graphs. Int J Comput Vis 95(3):213–239

    CrossRef  Google Scholar 

  34. Schuon S, Theobalt C, Davis J, Thrun S (2008) High-quality scanning using time-of-flight depth superresolution. In: IEEE Computer society conference on computer vision and pattern recognition workshops 2008

    Google Scholar 

  35. Schuon S, Theobalt C, Davis J, Thrun S (2009) Lidarboost depth superresolution for tof 3d shape scanning. In: CVPR 2009

    Google Scholar 

  36. Sharma A, Kumar S (2014) User-guided modulation of rendering techniques for detail inspection. In: GRAPP 2014—Proceedings of the 9th international conference on computer graphics theory and applications, Lisbon, Portugal, 5–8 January, 2014, pp 247–254

    Google Scholar 

  37. Shi J, Malik J (2000) Normalized cuts and image segmentation. IEEE Trans Pattern Anal Mach Intell 22(8):888–905

    CrossRef  Google Scholar 

  38. Slabaugh G, Schafer R (2004) Methods for volumetric reconstruction of visual scenes. Int J Comput Vis 179–199

    Google Scholar 

  39. Smisek J, Jancosek M, Pajdla T (2011) 3d with kinect. In: IEEE workshop on consumer depth cameras for computer vision

    Google Scholar 

  40. Snavely N, Seitz S, Szeliski R (2006) Photo tourism: exploring photo collections in 3d. In: Proceedings of ACM SIGGRAPH, pp 835–846

    Google Scholar 

  41. Snavely N, Seitz S, Szeliski R (2008) Modeling the world from internet photo collections. Int J Comput Vis 80(2):189–210

    CrossRef  Google Scholar 

  42. Snavely N, Seitz S, Szeliski R (2008) Skeletal graphs for efficient structure from motion. In: Proceedings of IEEE conference on computer vision and pattern recognition, pp 1–8

    Google Scholar 

  43. Sturm PF, Maybank SJ (1999) A method for interactive 3d reconstruction of piecewise planar objects from single images. Proc BMVC 1999:265–274

    Google Scholar 

  44. Trapp M, Semmo A, Pokorski R, Hermann CD, Dollner J, Eichhorn M, Heinzelmann M (2010) Communication of digtial cutltural heritage in public spaces by the example of roman cologne. In: Proceedings of Euro Mediterranean conference, pp 262–276

    Google Scholar 

  45. Triggs B, Mclauchlan P, Hartley R, Fitzgibbon A (2000) Bundle adjustment a modern synthesis. In: Vision algorithms: theory and practice, LNCS, pp 298–372

    Google Scholar 

  46. Wu C (2007) SiftGPU: a GPU implementation of scale invariant feature transform (SIFT). http://cs.unc.edu/~ccwu/siftgpu

  47. Wu C (2013) Towards linear-time incremental structure from motion. In: Proceedings of the international conference on 3D vision, 3DV ’13, pp 127–134

    Google Scholar 

  48. Wu C, Agarwal S, Curless B, Seitz S (2011) Multicore bundle adjustment. In: Proceedings of IEEE conference on computer vision and pattern recognition, pp 3057–3064

    Google Scholar 

  49. Yang Q, Yang R, Davis J, Nistr D (2007) Spatial-depth super resolution for range images. In: CVPR 2007

    Google Scholar 

Download references

Acknowledgements

The sponsorship and continuous support from Department of Science and Technology for the project are highly appreciated. The project has greatly benefitted from the collaboration of other institutes. In particular, BVBCET (Prof. Uma Mudenagudi), NIAS (Prof. Meera Natampally), IIIT Hyderabad (Prof. Anoop Namboodiri), IIT Bombay (Prof. Parag Chaudhuri), IISc Bangalore (Prof. Venu Madhav Govindu). The implementation and development required efforts of many research scholars and students. These efforts are parts of several Ph.D. and Masters theses. These include Brojeshwar Bhowmick (Ph.D.), Suvam Patra (Ph.D.), Nishant Bugalia (MSR 2016), Shantanu Chaudhari (M.Tech. 2016), Abhinav Shukla (M.Tech. 2011), Harsh Vardhan (M.Tech. 2011), Lissy Verma (M.Tech. 2011), Rahul Kumar (M.Tech. 2011), Nidhi Arora (M.Tech. 2011), Ankush Kumar (M.Tech. 2011), Anay Ghotikar (M.Tech. 2012), Ankit (M.Tech. 2012), Suvam Patra (M.Tech. 2012), Neeraj Kulkarni (M.Tech. 2012), Shruti Agarwal (M.Tech. 2012), Richa Gupta (M.Tech. 2013), Ramji Gupta (M.Tech. 2013), Kinshuk Sarabhai (MSR 2013) and Satyendra Singh (M.Tech. 2014).

Author information

Authors and Affiliations

  1. Indian Institute of Technology Delhi, New Delhi, India

    Prem Kalra, Subodh Kumar & Subhashis Banerjee

Authors
  1. Prem Kalra
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Subodh Kumar
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Subhashis Banerjee
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Prem Kalra .

Editor information

Editors and Affiliations

  1. Department of Electrical Engineering, Indian Institute of Technology, Hauz Khas, New Delhi, India

    Dr. Anupama Mallik

  2. CSIR-CEERI, Pilani and Department of Electrical Engineering, Indian Institute of Technology, Hauz Khas, New Delhi, India

    Prof. Santanu Chaudhury

  3. International Institute for Art, Culture and Democracy, Bengaluru, Karnataka, India

    Dr. Vijay Chandru

  4. National Institute of Advanced Studies, IISc Campus, Bengaluru, Karnataka, India

    Prof. Sharada Srinivasan

Rights and permissions

Reprints and Permissions

Copyright information

© 2017 Springer Nature Singapore Pte Ltd.

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Kalra, P., Kumar, S., Banerjee, S. (2017). Acquisition, Representation, Processing and Display of Digital Heritage Sites. In: Mallik, A., Chaudhury, S., Chandru, V., Srinivasan, S. (eds) Digital Hampi: Preserving Indian Cultural Heritage. Springer, Singapore. https://doi.org/10.1007/978-981-10-5738-0_6

Download citation

  • DOI: https://doi.org/10.1007/978-981-10-5738-0_6

  • Published:

  • Publisher Name: Springer, Singapore

  • Print ISBN: 978-981-10-5737-3

  • Online ISBN: 978-981-10-5738-0

  • eBook Packages: Computer ScienceComputer Science (R0)

search

Navigation