Abstract
We provide an approach to reconstruct spatiotemporal 3D models of aging objects such as fruit containing time-varying shape and appearance using multi-view time-lapse videos captured by a microenvironment of Raspberry Pi cameras. Our approach represents the 3D structure of the object prior to aging using a static 3D mesh reconstructed from multiple photographs of the object captured using a rotating camera track. We manually align the 3D mesh to the images at the first time instant. Our approach automatically deforms the aligned 3D mesh to match the object across the multi-viewpoint time-lapse videos. We texture map the deformed 3D meshes with intensities from the frames at each time instant to create the spatiotemporal 3D model of the object. Our results reveal the time dependence of volume loss due to transpiration and color transformation due to enzymatic browning on banana peels and in exposed parts of bitten fruit.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsNotes
- 1.
In \(\begin{bmatrix}\mathbf {x}^0_{i_j}; 1\end{bmatrix}\), the semi-colon appends 1 at the end of column vector \(\mathbf {x}^0_{i_j}\).
References
Adobe: ReMake. https://remake.autodesk.com/about/
de Aguiar, E., Stoll, C., Theobalt, C., Ahmed, N., Seidel, H.P., Thrun, S.: Performance capture from sparse multi-view video. ACM Trans. Graph. 25(3) (2008)
Baraff, D., Witkin, A.: Dynamic simulation of non-penetrating flexible bodies. In: SIGGRAPH (1992)
Baraff, D., Witkin, A.: Large steps in cloth simulation. In: SIGGRAPH (1998)
Baran, I., Popović, J.: Automatic rigging and animation of 3D characters. ACM Trans. Graph. 26(3) (2007)
Beeler, T., Hahn, F., Bradley, D., Bickel, B., Beardsley, P., Gotsman, C., Sumner, R.W., Gross, M.: High-quality passive facial performance capture using anchor frames. In: SIGGRAPH (2011)
Chuang, M., Luo, L., Brown, B.J., Rusinkiewicz, S., Kazhdan, M.: Estimating the Laplace-Beltrami operator by restricting 3D functions. In: CGF (2009)
Collet, A., Berenson, D., Srinivasa, S.S., Ferguson, D.: Object recognition and full pose registration from a single image for robotic manipulation. In: ICRA (2009)
Debevec, P.: Rendering synthetic objects into real scenes: bridging traditional and image-based graphics with global illumination and high dynamic range photography. In: SIGGRAPH Classes (2008)
Dorsey, J., Pedersen, H.K., Hanrahan, P.: Flow and changes in appearance. In: SIGGRAPH Courses, p. 3 (2005)
Enrique, S., Koudelka, M., Belhumeur, P., Dorsey, J., Nayar, S., Ramamoorthi, R.: Time-varying textures: definition, acquisition, and synthesis. In: SIGGRAPH Sketches (2005)
Fischler, M.A., Bolles, R.C.: Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartoraphy. CACM 24(6), 381–395 (1981)
Gu, J., Tu, C.I., Ramamoorthi, R., Belhumeur, P., Matusik, W., Nayar, S.: Time-varying surface appearance: acquisition, modeling and rendering. ACM Trans. Graph. 25(3) (2006)
Guo, L., Quant, H., Lamb, N., Lowit, B., Banerjee, N.K., Banerjee, S.: Multi-camera microenvironment to capture multi-view time-lapse videos for 3D analysis of aging objects. In: International Conference on Multimedia Modeling (2018)
Jackson, A.S., Bulat, A., Argyriou, V., Tzimiropoulos, G.: Large pose 3D face reconstruction from a single image via direct volumetric CNN regression. arXiv preprint arXiv:1703.07834 (2017)
Kholgade, N., Simon, T., Efros, A., Sheikh, Y.: 3D object manipulation in a single photograph using stock 3D models. In: SIGGRAPH (2014)
Kider, J.T., Raja, S., Badler, N.I.: Fruit senescence and decay simulation. In: CG Forum (2011)
Lamb, N., Banerjee, N.K., Banerjee, S.: Programmatic 3D printing of a revolving camera track to automatically capture dense images for 3D scanning of objects. In: International Conference on Multimedia Modeling (2018)
Langenbucher, T., Merzbach, S., Möller, D., Ochmann, S., Vock, R., Warnecke, W., Zschippig, M.: Time-varying BTFs. In: CESCG (2010)
Lepetit, V., Moreno-Noguer, F., Fua, P.: EPnP: an accurate O(n) solution to the PnP problem. IJCV 81(2), 155–166 (2009)
Li, Y., Fan, X., Mitra, N.J., Chamovitz, D., Cohen-Or, D., Chen, B.: Analyzing growing plants from 4D point cloud data. ACM Trans. Graph. 32(6) (2013)
Lim, J.J., Khosla, A., Torralba, A.: FPM: fine pose parts-based model with 3D CAD models. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8694, pp. 478–493. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-10599-4_31
Lu, J., Georghiades, A.S., Glaser, A., Wu, H., Wei, L.Y., Guo, B., Dorsey, J., Rushmeier, H.: Context-aware textures. ACM Trans. Graph. 26(1) (2007)
Lucas, B.D., Kanade, T.: An iterative image registration technique with an application to stereo vision. In: IJCAI (1981)
Nicolas, J.J., Richard-Forget, F.C., Goupy, P.M., Amiot, M.J., Aubert, S.Y.: Enzymatic browning reactions in apple and apple products. Critical Rev. Food Sci. Nutr. 34(2), 109–157 (1994)
Paquette, E., Poulin, P., Drettakis, G.: The simulation of paint cracking and peeling. In: GI (2002)
Quant, H., Banerjee, S., Banerjee, N.K.: A virtual reality interface for interactions with spatiotemporal 3D data. In: International Conference on Multimedia Modeling (2018)
Ramamoorthi, R., Hanrahan, P.: An efficient representation for irradiance environment maps. In: SIGGRAPH (2001)
Sorkine, O., Alexa, M.: As-rigid-as-possible surface modeling. In: EUROGRAPHICS (2007)
Yin, X., Fujimoto, T., Chiba, N., Tanaka, H.T.: Modeling of wood aging caused by biological deterioration. JACIII 22(2), 125–131 (2008)
Zhang, C., Chen, T.: Efficient feature extraction for 2D/3D objects in mesh representation. In: ICIP (2001)
Acknowledgements
This work was partially supported by the National Science Foundation (NSF) grant #1730183.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer International Publishing AG
About this paper
Cite this paper
Guo, L., Quant, H., Lamb, N., Lowit, B., Banerjee, S., Banerjee, N.K. (2018). Spatiotemporal 3D Models of Aging Fruit from Multi-view Time-Lapse Videos. In: Schoeffmann, K., et al. MultiMedia Modeling. MMM 2018. Lecture Notes in Computer Science(), vol 10704. Springer, Cham. https://doi.org/10.1007/978-3-319-73603-7_38
Download citation
DOI: https://doi.org/10.1007/978-3-319-73603-7_38
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-73602-0
Online ISBN: 978-3-319-73603-7
eBook Packages: Computer ScienceComputer Science (R0)