Abstract
Geometric morphometrics is the statistical analysis of landmark-based shape variation and its covariation with other variables. Over the past two decades, the gold standard of landmark data acquisition has been manual detection by a single observer. This approach has proven accurate and reliable in small-scale investigations. However, big data initiatives are increasingly common in biology and morphometrics. This requires fast, automated, and standardized data collection. We combine techniques from image registration, geometric morphometrics, and deep learning to automate and optimize anatomical landmark detection. We test our method on high-resolution, micro-computed tomography images of adult mouse skulls. To ensure generalizability, we use a morphologically diverse sample and implement fundamentally different deformable registration algorithms. Compared to landmarks derived from conventional image registration workflows, our optimized landmark data show up to a 39.1% reduction in average coordinate error and a 36.7% reduction in total distribution error. In addition, our landmark optimization produces estimates of the sample mean shape and variance–covariance structure that are statistically indistinguishable from expert manual estimates. For biological imaging datasets and morphometric research questions, our approach can eliminate the time and subjectivity of manual landmark detection whilst retaining the biological integrity of these expert annotations.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Data Availability
Our training and testing images are available at https://www.facebase.org.
Code Availability
Our code is freely available at https://github.com/jaydevine/Landmarking.
References
Adams, D. C., Rohlf, F. J., & Slice, D. E. (2004). Geometric morphometrics: Ten years of progress following the ‘revolution’. Italian Journal of Zoology,71(1), 5–16. https://doi.org/10.1080/11250000409356545.
Adams, D. C., Rohlf, F. J., & Slice, D. E. (2013). A field comes of age: Geometric morphometrics in the 21st century. Hystrix,24(1), 7–14. https://doi.org/10.4404/hystrix-24.1-6283.
Attanasio, C., Nord, A. S., Zhu, Y., Blow, M. J., Li, Z., Denise, K., et al. (2014). Fine tuning of craniofacial morphology by distant-acting enhancers. Science,342(6157), 1–20. https://doi.org/10.1126/science.1241006.
Avants, B. B., Tustison, N. J., Song, G., Cook, P. A., Klein, A., & Gee, J. C. (2011). A reproducible evaluation of ANTs similarity metric performance in brain image registration. NeuroImage,54(3), 2033–2044. https://doi.org/10.1016/j.neuroimage.2010.09.025.
Bates, D., Mächler, M., Bolker, B., & Walker, S. (2015). Fitting linear mixed-effects models using lme4. Journal of Statistical Software,67(1), 1–48. https://doi.org/10.18637/jss.v067.i01.
Bezanson, J., Edelman, A., Karpinski, S., & Shah, V. B. (2017). Julia: A fresh approach to numerical computing. SIAM Review,59(1), 65–98. https://doi.org/10.1137/141000671.
Bookstein, F. L. (1989). Principal warps: Thin-plate splines and the decomposition of deformations. IEEE Transactions on Pattern Analysis and Machine Intelligence,11(6), 567–585.
Bookstein, F. L. (1991). Morphometric tools for landmark data: Geometry and biology. Cambridge: Cambridge University Press.
Bookstein, F. L., & Mitteroecker, P. (2014). Comparing covariance matrices by relative eigenanalysis, with applications to organismal biology. Evolutionary Biology,41(2), 336–350. https://doi.org/10.1007/s11692-013-9260-5.
Bowman, A. W., & Azzalini, A. (1997). Applied smoothing techniques for data analysis: The Kernel approach with S-Plus illustrations. New York: Oxford University Press.
Bromiley, P. A., Schunke, A. C., Ragheb, H., Thacker, N. A., & Tautz, D. (2014). Semi-automatic landmark point annotation for geometric morphometrics. Frontiers in Zoology,11(1), 61. https://doi.org/10.1186/s12983-014-0061-1.
Bycroft, C., Freeman, C., Petkova, D., Band, G., Elliott, L. T., Sharp, K., et al. (2018). The UK biobank resource with deep phenotyping and genomic data. Nature,562(7726), 203. https://doi.org/10.1038/s41586-018-0579-z.
Collins, D. L., & Evans, A. C. (1997). ANIMAL: Validation and applications of nonlinear registration-based segmentation. International Journal of Pattern Recognition and Artificial Intelligence,11(8), 1271–1294.
Collyer, M. L., & Adams, D. C. (2018). RRPP: An R package for fitting linear models high-dimensional data using residual randomization. Methods in Ecology and Evolution,9(7), 1772–1779. https://doi.org/10.1111/2041-210X.13029.
Collyer, M. L., & Adams, D. C. (2019). RRPP: Linear model evaluation with randomized residuals in a permutation procedure. R package version 0.4.0. [WWW Document]. Retrieved from https://cran.r-project.org/package=RRPP.
Duchon, J. (1976). Interpolation Des Fonctions De Deux Variables Suivant Le Principe De La Flexion Des Plaques Minces. Analyse Numérique,10(R3), 5–12.
Dryden, I. L. (2018). Shapes: Statistical shape analysis. R package version 1.2.4. [WWW Document]. Retrieved from https://CRAN.R-project.org/package=shapes.
Dryden, I. L., Koloydenko, A., & Zhou, D. (2009). Non-Euclidean statistics for covariance matrices, with applications to diffusion tensor imaging. The Annals of Applied Statistics,3(3), 1102–1123. https://doi.org/10.1214/09-AOAS249.
Dryden, I. L., & Mardia, K. V. (1998). Statistical shape analysis. London: Wiley.
Evans, A. C., Janke, A. L., Collins, D. L., & Baillet, S. (2012). Brain templates and atlases. NeuroImage,62(2), 911–922. https://doi.org/10.1016/j.neuroimage.2012.01.024.
Freimer, N., & Sabatti, C. (2003). The human phenome project. Nature Genetics,34(1), 15–21.
Friedel, M., van Eede, M. C., Pipitone, J., Chakravarty, M. M., & Lerch, J. P. (2014). Pydpiper: A flexible toolkit for constructing novel registration pipelines. Frontiers in Neuroinformatics,8, 67. https://doi.org/10.3389/fninf.2014.00067.
Gower, J. C. (1975). Generalized procrustes analysis. Psychometrika,40(1), 33–50.
Hallgrímsson, B., Willmore, K., Dorval, C., & Cooper, D. M. L. (2004). Craniofacial variability and modularity in macaques and mice. Journal of Experimental Zoology Part B,302(3), 207–225. https://doi.org/10.1002/jez.b.21002.
Hallgrímsson, B., Brown, J. J., Ford-Hutchinson, A. F., Sheets, H. D., Zelditch, M. L., & Jirik, F. R. (2006). The brachymorph mouse and the developmental-genetic basis for canalization and morphological integration. Evolution & Development,8(1), 61–73. https://doi.org/10.1111/j.1525-142X.2006.05075.x.
Hallgrímsson, B., Jamniczky, H., Young, N. M., Rolian, C., Parsons, T. E., Boughner, J. C., et al. (2009a). Deciphering the palimpsest: Studying the relationship between morphological integration and phenotypic covariation. Evolutionary Biology,36(4), 355–376. https://doi.org/10.1007/s11692-009-9076-5.
Hallgrímsson, B., Boughner, J. C., Turinsky, A. L., & Sensen, C. W. (2009b). Geometric morphometrics and the study of development. In C. W. Sensen & B. Hallgrímsson (Eds.), Advanced imaging in biology and medicine (pp. 319–336). Berlin: Springer-Verlag.
Hallgrímsson, B., Green, R. M., Katz, D. C., Fish, J. L., Bernier, F. P., Roseman, C. C., et al. (2019a). The developmental-genetics of canalization. Seminars in Cell and Developmental Biology,88, 67–79. https://doi.org/10.1016/j.semcdb.2018.05.019.
Hallgrímsson, B., Katz, D. C., Aponte, J. D., Larson, J. R., Devine, J., Gonzalez, P. N., et al. (2019b). Integration and the developmental genetics of allometry. Integrative and Comparative Biology,59(5), 1369–1381. https://doi.org/10.1093/icb/icz105.
Heckemann, R. A., Hajnal, J. V., Aljabar, P., Rueckert, D., & Hammers, A. (2006). Automatic anatomical brain MRI segmentation combining label propagation and decision fusion. NeuroImage,33(1), 115–126. https://doi.org/10.1016/j.neuroimage.2006.05.061.
Houle, D., Govindaraju, D. R., & Omholt, S. (2010). Phenomics: The next challenge. Nature Reviews Genetics,11(12), 855–866. https://doi.org/10.1038/nrg2897.
Innes, M. (2018). Flux: Elegant machine learning with Julia. Journal of Open Source Software,3(25), 602. https://doi.org/10.21105/joss.00602.
Kendall, D. G. (1984). Shape manifolds, procrustean metrics and complex projective spaces. Bulletin of the London Mathematical Society,16(2), 81–121.
Kingma, D. P., & Ba, J. L. (2015). Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980.
Klein, A., Andersson, J., Ardekani, B. A., Ashburner, J., Avants, B., Chiang, M. C., et al. (2009). Evaluation of 14 nonlinear deformation algorithms applied to human brain MRI registration. NeuroImage,46(3), 786–802. https://doi.org/10.1016/j.neuroimage.2008.12.037.
Klingenberg, C. P. (2002). Morphometrics and the role of the phenotype in studies of the evolution of developmental mechanisms. Gene,287(1–2), 3–10. https://doi.org/10.1016/S0378-1119(01)00867-8.
Klingenberg, C. P. (2008). Morphological integration and developmental modularity. The Annual Review of Ecology, Evolution, and Systematics,39, 115–132. https://doi.org/10.1146/annurev.ecolsys.37.091305.110054.
Klingenberg, C. P. (2009). Morphometric integration and modularity in configurations of landmarks: Tools for evaluating a priori hypotheses. Evolution & Development,11(4), 405–421. https://doi.org/10.1111/j.1525-142X.2009.00347.
Le Maître, A., & Mitteroecker, P. (2019). Multivariate comparison of variance in R. Methods in Ecology and Evolution,10(9), 1380–1392. https://doi.org/10.1111/2041-210X.13253.
Li, M., Cole, J. B., Manyama, M., Larson, J. R., Liberton, D. K., Riccardi, S. L., et al. (2017). Rapid automated landmarking for morphometric analysis of three-dimensional facial scans. Journal of Anatomy,230(4), 607–618. https://doi.org/10.1111/joa.12576.
Lieberman, D. E., Hallgrímsson, B., Liu, W., Parsons, T. E., & Jamniczky, H. A. (2008). Spatial packing, cranial base angulation, and craniofacial shape variation in the mammalian skull: Testing a new model using mice. Journal of Anatomy,212(6), 720–735. https://doi.org/10.1111/j.1469-7580.2008.00900.x.
Maga, A. M., Tustison, N. J., & Avants, B. B. (2017). A population level atlas of mus musculus craniofacial skeleton and automated image-based shape analysis. Journal of Anatomy,231(3), 433–443. https://doi.org/10.1111/joa.12645.
Mazziotta, J., Toga, A., Evans, A., Fox, P., Lancaster, J., Zilles, K., et al. (2001). A probabilistic atlas and reference system for the human brain: International Consortium for Brain Mapping (ICBM). Philosophical Transactions of the Royal Society B,356, 1293–1322. https://doi.org/10.1098/rstb.2001.0915.
Mitteroecker, P., & Gunz, P. (2009). Advances in geometric morphometrics. Evolutionary Biology,36(2), 235–247. https://doi.org/10.1007/s11692-009-9055-x.
Nelder, J. A., & Mead, R. (1965). A simplex method for function minimization. The Computer Journal,7(4), 308–313.
Oksanen, J., Blanchet, F. G., Friendly, M., Kindt, R., Legendre, P., McGlinn, D., Minchin, P. R., et al. (2019). vegan: Community ecology package. R package version 2.5–4. [WWW Document]. Retrieved from https://cran.r-project.org/package=vegan.
Oliveira, F. P., & Tavares, J. M. R. (2014). Medical image registration: A review. Computer Methods in Biomechanics,17(2), 73–93. https://doi.org/10.1080/10255842.2012.670855.
O'Higgins, P., Chadfield, P., & Jones, N. (2001). Facial growth and the ontogeny of morphological variation within and between the primates cebus Apella and Cercocebus Torquatus. Journal of Zoology,254(3), 337–357.
Percival, C. J., Green, R., Marcucio, R., & Hallgrímsson, B. (2014). Surface landmark quantification of embryonic mouse craniofacial morphogenesis. BMC Developmental Biology,14(1), 31. https://doi.org/10.1186/1471-213X-14-31.
Percival, C. J., Devine, J., Darwin, B. C., Liu, W., van Eede, M., Henkelman, R. M., et al. (2019). The effect of automated landmark identification on morphometric analyses. Journal of Anatomy,234(6), 917–935. https://doi.org/10.1111/joa.12973.
Raup, D. M. (1966). Geometric analysis of shell coiling. Journal of Paleontology,40(5), 1178–1190.
R Core Team. (2018). R: A language and environment for statistical computing. Vienna: R Foundation for Statistical Computing. [WWW Document]. Retrieved from https://www.R-project.org/.
Robinson, P. N. (2012). Deep phenotyping for precision medicine. Human Mutation,3395, 777–780. https://doi.org/10.1002/humu.22080.
Rohlf, F. J. (2003). Bias and error in estimates of mean shape in geometric morphometrics. Journal of Human Evolution,44(6), 665–683. https://doi.org/10.1016/S0047-2484(03)00047-2.
Rohlf, F. J., & Slice, D. (1990). Extensions of the procrustes method for the optimal superimposition of landmarks. Systematic Biology,39(1), 40–59.
Rohlfing, T., & Maurer, C. R. (2006). Shape-based averaging. IEEE Transactions on Image Processing,16(1), 153–161.
Rueckert, D., Sonoda, L. I., Hayes, C., Hill, D. L. G., Leach, M. O., & Hawkes, D. J. (1999). Nonrigid registration using free-form deformations: Application to breast MR images. IEEE Transactions on Medical Imaging,18(8), 712–721.
Schlager, S. (2017). Morpho and Rvcg – shape analysis in R: R-packages for geometric morphometrics, shape analysis and surface manipulations. In G. Zheng, S. Li, & G. Szekely (Eds.), Statistical shape and deformation analysis (pp. 217–256). Cambridge: Academic Press.
Schork, N. J. (1997). Genetics of complex disease: Approaches, problems, and solutions. The American Journal of Respiratory and Critical Care,156(4), S103–S109.
Sotiras, A., Davatzikos, C., & Paragios, N. (2013). Deformable medical image registration: A survey. IEEE Transactions on Medical Imaging,32(7), 1153–1190.
Vincent, R. D., Neelin, P., Khalili-Mahani, N., Janke, A. L., Fonov, V. S., Robbins, S. M., et al. (2016). MINC 2.0: A flexible format for multi-modal images. Frontiers in Neuroinformatics,10, 35. https://doi.org/10.3389/fninf.2016.00035.
von Cramon-Taubadel, N., Frazier, B. C., & Lahr, M. M. (2007). The problem of assessing landmark error in geometric morphometrics: Theory, methods, and modifications. American Journal of Physical Anthropology,134(1), 24–35. https://doi.org/10.1002/ajpa.20616.
Vora, S. R., Camci, E. D., & Cox, T. C. (2016). Postnatal ontogeny of the cranial base and craniofacial skeleton in male C57BL/6J mice: A reference standard for quantitative analysis. Frontiers in Physiology,6, 417. https://doi.org/10.3389/fphys.2015.00417.
Wang, H., Suh, J. W., Das, S. R., Pluta, J. B., & Yushkevich, P. A. (2012). Multi-atlas segmentation with joint label fusion. IEEE Transactions on Pattern Analysis and Machine Intelligence,35(3), 611–623. https://doi.org/10.1109/TPAMI.2012.143.
Wei, T., & Simko, V. (2017). R package "corrplot": Visualization of a correlation matrix. R package version 0.84. [WWW Document]. Retrieved from https://github.com/taiyun/corrplot.
Young, R., & Maga, A. M. (2015). Performance of single and multi-atlas based automated landmarking methods compared to expert annotations in volumetric microCT datasets of mouse mandibles. Frontiers in Zoology,12(1), 33. https://doi.org/10.1186/s12983-015-0127-8.
Zelditch, M. L., Swiderski, D. L., Sheets, H. D., & Fink, W. L. (2012). Geometric morphometrics for biologists: A primer. San Diego: Elsevier Academic Press.
Funding
This work was supported by National Institutes of Health R01 01DE019638 to BH and RM, the Canadian Institutes of Health Research Foundation grant, the Natural Sciences and Engineering Research Council Grant 238992–17, and the Canadian Foundation for Innovation Grant #36262 to BH.
Author information
Authors and Affiliations
Contributions
JD, JDA, DCK, CJP, and BH conceived the ideas and designed methodology. JD, JDA, and WL collected the data. JD, JDA, DCK, and BH analyzed the data. JD, DCK, LDLV, NDF, RM, CJP, and BH contributed critically to the manuscript drafts and edits. All authors gave final approval for publication.
Corresponding author
Ethics declarations
Conflict of interest
The authors have no conflict of interest to declare.
Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
About this article
Cite this article
Devine, J., Aponte, J.D., Katz, D.C. et al. A Registration and Deep Learning Approach to Automated Landmark Detection for Geometric Morphometrics. Evol Biol 47, 246–259 (2020). https://doi.org/10.1007/s11692-020-09508-8
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11692-020-09508-8