Single Image Plankton 3D Reconstruction from Extended Depth of Field Shadowgraph
Marine plankton occurs in the ocean with strongly varying degrees of sparsity. For in-situ plankton measurements the shadowgraph has been established as the observation device of choice in recent years. In this paper a novel depth from defocus based approach to partially coherent 3D reconstruction of marine plankton volumes is presented. With a combination of recent advances in coherent image restoration and deep learning, we create a 3D view of the shadowgraph observation volume. For the selection of in-focus images we develop a novel training data generation technique. This kind of reconstruction was previously only possible with holographic imaging systems, which require laser illumination with high coherence, which often causes parasitic interferences on optical components and speckles. The new 3D visualization gives easily manageable data by resulting in a sharp view of each plankton together with its depth and position. Moreover, this approach allows the creation of all-in-focus images of larger observation volumes, which is otherwise impossible due to the physically limited depth of field. We show an effective increase in depth of field by a factor of 7, which allows marine researchers to use larger observation volumes and thus a much more effective observation of marine plankton.
KeywordsMarine plankton Shadowgraph Image restoration
This work has partly been supported by the German Research Foundation (DFG) Cluster of Excellence FUTURE OCEAN under proposals CP1331 and CP1525 and by the Petersen-Foundation in Kiel under project 385.
- 1.Alvarez-Palacio, D., Garcia-Sucerquia, J.: Digital in-line holographic microscopy with partially coherent light: micrometer resolution. Revista mexicana de física 56(6), 445–448 (2010)Google Scholar
- 5.Eigen, D., Puhrsch, C., Fergus, R.: Depth map prediction from a single image using a multi-scale deep network. In: Advances in Neural Information Processing Systems, pp. 2366–2374 (2014)Google Scholar
- 6.Harris, R., Wiebe, P., Lenz, J., Skjoldal, H.R., Huntley, M.: ICES Zooplankton Methodology Manual. Academic press, London (2000)Google Scholar
- 7.He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 770–778 (2016)Google Scholar
- 9.Krizhevsky, A., Sutskever, I., Hinton, G.E.: ImageNet classification with deep convolutional neural networks. In: Advances in Neural Information Processing Systems, pp. 1097–1105 (2012)Google Scholar
- 10.Levin, A., Weiss, Y., Durand, F., Freeman, W.T.: Understanding and evaluating blind deconvolution algorithms. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 1964–1971 (2009)Google Scholar
- 11.Mazumdar, A.: Principles and techniques of schlieren imaging systems. Columbia University Computer Science Technical reports (2013)Google Scholar
- 13.Pitois, S.G., Tilbury, J., Bouch, P., Close, H., Barnett, S., Culverhouse, P.F.: Comparison of a cost-effective integrated plankton sampling and imaging instrument with traditional systems for mesozooplankton sampling in the celtic sea. Front. Marine Sci. 5, 5 (2018). https://doi.org/10.3389/fmars.2018.00005CrossRefGoogle Scholar
- 17.Szegedy, C., et al.: Going deeper with convolutions. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1–9 (2015)Google Scholar