Abstract
Purpose
A better understanding of photometry in laparoscopic images can increase the reliability of computer-assisted surgery applications. Photometry requires modelling illumination, tissue reflectance and camera response. There exists a large variety of light models, but no systematic and reproducible evaluation. We present a review of light models in laparoscopic surgery, a unified calibration approach, an evaluation methodology, and a practical use of photometry.
Method
We use images of a calibration checkerboard to calibrate the light models. We then use these models in a proposed dense stereo algorithm exploiting the shading and simultaneously extracting the tissue albedo, which we call dense shading stereo. The approach works with a broad range of light models, giving us a way to test their respective merits.
Results
We show that overly complex light models are usually not needed and that the light source position must be calibrated. We also show that dense shading stereo outperforms existing methods, in terms of both geometric and photometric errors, and achieves sub-millimeter accuracy.
Conclusion
This work demonstrates the importance of careful light modelling and calibration for computer-assisted surgical applications. It gives guidelines on choosing the best performing light model.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Notes
References
Agarwal S, Mierle K (2012) Ceres solver. http://ceres-solver.org
Aggarwal M, Hua H, Ahuja N (2001) On cosine-fourth and vignetting effects in real lenses. In: Proceedings eighth IEEE international conference on computer vision. ICCV 2001, vol 1, pp 472–479. IEEE
Bernhardt S, Nicolau SA, Bartoli A, Agnus V, Soler L, Doignon C (2015) Using shading to register an intraoperative CT scan to a laparoscopic image. In: CARE, pp 59–68. Springer
Collins T, Bartoli A (2012) 3D reconstruction in laparoscopy with close-range photometric stereo. In: MICCAI, pp 634–642. Springer
Collins T, Bartoli A (2012) Towards live monocular 3D laparoscopy using shading and specularity information. In: IPCAI
Collins T, Bartoli A, Bourdel N, Canis M (2016) Robust, real-time, dense and deformable 3D organ tracking in laparoscopic videos. In: MICCAI
Forster CQ, Tozzi CL (2000) Towards 3d reconstruction of endoscope images using shape from shading. In: Proceedings 13th Brazilian symposium on computer graphics and image processing (Cat. No. PR00878), pp 90–96. IEEE
Gallardo M, Collins T, Bartoli A (2017) Dense non-rigid structure-from-motion and shading with unknown albedos. In: ICCV
Garrido-Jurado S, Muñoz-Salinas R, Madrid-Cuevas FJ, Marín-Jiménez MJ (2014) Automatic generation and detection of highly reliable fiducial markers under occlusion. Pattern Recognit 47(6):2280–2292
Koo B, Özgür E, Le Roy B, Buc E, Bartoli A (2017) Deformable registration of a preoperative 3D liver volume to a laparoscopy image using contour and shading cues. In: MICCAI
Land EH, McCann JJ (1971) Lightness and retinex theory. Josa 61(1):1–11
Langguth F, Sunkavalli K, Hadap S, Goesele M (2016) Shading-aware multi-view stereo. In: ECCV
Liu-Yin Q, Yu R, Agapito L, Fitzgibbon A, Russell C (2017) Better together: joint reasoning for non-rigid 3D reconstruction with specularities and shading. arXiv preprint arXiv:1708.01654
Mahmoud N, Cirauqui I, Hostettler A, Doignon C, Soler L, Marescaux J, Montiel J (2016) Orbslam-based endoscope tracking and 3D reconstruction. In: International workshop on computer-assisted and robotic endoscopy, pp 72–83. Springer
Malis E, Vargas M (2007) Deeper understanding of the homography decomposition for vision-based control. Technical Report 6303, INRIA. Available from: http://hal.inria.fr/inria-00174036/fr/
Malti A, Bartoli A (2014) Combining conformal deformation and Cook–Torrance shading for 3-D reconstruction in laparoscopy. IEEE Trans Biomed Eng 61(6):1684–1692
Mariyanayagam D, Gurdjos P, Chambon S, Brunet F, Charvillat V (2018) Pose estimation of a single circle using default intrinsic calibration. In: ACCV
Okatani T, Deguchi K (1997) Shape reconstruction from an endoscope image by shape from shading technique for a point light source at the projection center. Comput Vis Image Underst 66(2):119–131
Parot V, Lim D, González G, Traverso G, Nishioka NS, Vakoc BJ, Durr NJ (2013) Photometric stereo endoscopy. J Biomed Opt 18(7):076017
Stoyanov D, Darzi A, Yang GZ (2004) Dense 3d depth recovery for soft tissue deformation during robotically assisted laparoscopic surgery. In: International conference on medical image computing and computer-assisted intervention, pp 41–48. Springer
Wang Z, Bovik AC, Sheikh HR, Simoncelli EP (2004) Image quality assessment: from error visibility to structural similarity. IEEE Trans Image Process 13(4):600–612
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The authors declare that they have no conflict of interest.
Animals performed
This chapter does not contain any studies with animals performed by any of the authors.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
About this article
Cite this article
Modrzejewski, R., Collins, T., Hostettler, A. et al. Light modelling and calibration in laparoscopy. Int J CARS 15, 859–866 (2020). https://doi.org/10.1007/s11548-020-02161-8
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11548-020-02161-8