A Multi-Image Shape-from-Shading Framework for Near-Lighting Perspective Endoscopes
This article formulates a near-lighting shape-from-shading problem with a pinhole camera (perspective projection) and presents a solution to reconstruct the Lambertian surface of bones using a sequence of overlapped endoscopic images, with partial boundaries in each image. First we extend the shape-from-shading problem to deal with perspective projection and near point light sources that are not co-located with the camera center. Secondly we propose a multi-image framework which can align partial shapes obtained from different images in the world coordinates by tracking the endoscope. An iterative closest point (ICP) algorithm is used to improve the matching and recover complete occluding boundaries of the bone. Finally, a complete and consistent shape is obtained by simultaneously re-growing the surface normals and depths in all views. In order to fulfill our shape-from-shading algorithm, we also calibrate both geometry and photometry for an oblique-viewing endoscope that are not well addressed before in the previous literatures. We demonstrate the accuracy of our technique using simulations and experiments with artificial bones.
KeywordsMulti-image Shape-from-shading Near-lighting Perspective projection Calibration Endoscope Bone
Unable to display preview. Download preview PDF.
- Clarkson, M. J., Rueckert, D., King, A. P., Edwards, P. J., Hill, D. L. G., & Hawkes, D. J. (1999). Registration of video images to tomographic images by optimising mutual information using texture mapping. In LNCS: Vol. 1679. Proceedings of the second international conference on medical image computing and computer-assisted intervention (MICCAI’99) (pp. 579–588). Berlin: Springer. CrossRefGoogle Scholar
- Forster, C. H. Q., & Tozzi, C. L. (2000). Toward 3d reconstruction of endoscope images using shape from shading. In Proceedings of the 13th Brazilian symposium on computer graphics and image processing (SIBGRAPHI’00) (pp. 90–96). Google Scholar
- Fuchs, H., Livingston, M. A., Raskar, R., Colucci, D., Keller, K., State, A., Crawford, J. R., Rademacher, P., Drake, S. H., & Meyer, A. A. (1998). Augmented reality visualization for laparoscopic surgery. In LNCS: Vol. 1496. Proceedings of the first international conference on medical image computing and computer-assisted intervention (MICCAI’98) (pp. 934–943). Berlin: Springer. Google Scholar
- Horn, B. K. P., & Brooks, M. J. (1989). Shape from shading. Cambridge: MIT. Google Scholar
- Kozera, R., & Noakes, L. (2004). Noise reduction in photometric stereo with non-distant light sources. Proceedings of the International Conference on Computer Vision and Graphics (ICCVG’04), 32, 103–110. Google Scholar
- Leclerc, Y. G., & Bobick, A. F. (1991). The direct computation of height from shading. In Proceedings of IEEE computer society conference on computer vision and pattern recognition (CVPR’91) (pp. 552–558). Google Scholar
- Mourgues, F., Devernay, F., Malandain, G., & Coste-Manière, E. (2001). 3d reconstruction of the operating field for image overlay in 3d-endoscopic surgery. In: Proceedings of the IEEE and ACM international symposium on augmented reality (ISAR’01) (pp. 191–192). Google Scholar
- Prados, E., & Faugeras, O. (2005). Shape from shading: a well-posed problem? Proceedings of the 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’05), 2, 870–877. Google Scholar
- Seshamani, S., Lau, W., & Hager, G. (2006). Real-time endoscopic mosaicking. In LNCS: Vol. 4190. Proceedings of the Ninth International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI’06) (pp. 355–363). Berlin: Springer. Google Scholar
- Woodham, R. J. (1980). Photometric method for determining surface orientation from multiple images. Optical Engineering, 19(1), 139–144. Google Scholar
- Zhang, Z. (1998). A flexible new technique for camera calibration (Technical Report MSR-TR-98-71). Microsoft Research. Google Scholar