Towards Effective Diagnosis and Prediction via 3D Patient Model: A Complete Research Plan

  • Nadia Magnenat Thalmann
  • Hon Fai Choi
  • Daniel Thalmann
Chapter
First Online:

Abstract

Healthcare can be brought to a principally new level by creating virtual bodies of real patients and using them for diagnosis and treatment planning along with the real patients. It will help to improve the quality of healthcare services, and reduce the healthcare cost, especially when dealing with the rise in musculoskeletal disorders in the aging population. However, such an approach needs further investigation and development of innovative technologies in order to meet the challenging requirements for medical application. Virtual models of patients must offer the possibility to be individualized with complex subject-specific data but also enable a comprehensive visualization and analysis necessary for a reliable diagnosis and follow-up. The creation of this virtualization consists of different main components, which are automated anatomical extraction and visualization from medical images, computational musculoskeletal simulations, building models for diagnosis and medical education and validation and evaluation of modeling methodologies. The challenges, current solutions and a possible strategy for future innovations are presented in this chapter.

Keywords

Virtual patient  3D model MRI segmentation Musculoskeletal simulation  Surgical planning 
This is a preview of subscription content, log in to check access.

Notes

Acknowledgments

This work was supported by the European Marie Curie Initial Training Network MultiScaleHuman (FP7-PEOPLE-2011-ITN-289897).

References

  1. 1.
    Wilson, W., van Donkelaar, C. C., van Rietbergen, B., & Huiskes, R. (2005). A fibril-reinforced poroviscoelastic swelling model for articular cartilage. Journal of Biomechanics, 38, 1195–1204.CrossRefGoogle Scholar
  2. 2.
    Cowin, S. C., & Doty, S. B. (2007). Tissue Mechanics. New York: Springer.CrossRefMATHGoogle Scholar
  3. 3.
    Mow, V. C., & Guo, X. E. (2002). Mechano-electrochemical properties of articular cartilage. Annual Review of Biomedical Engineering, 4, 175–209.CrossRefGoogle Scholar
  4. 4.
    Fung, Y. C. (1993). Biomechanics: Mechanical properties of living tissues (2nd ed.). New York: Springer.Google Scholar
  5. 5.
    Laasanen, M. S., et al. (2003). Biomechanical properties of knee articular cartilage. Biorheology, 40, 133–140.Google Scholar
  6. 6.
    Fortis, A. P., Kostopoulos, V., Panagiotopoulos, E., Tsantzalis, S., & Kokkinos, A. (2004). Viscoelastic properties of cartilage-subchondral bone complex in osteoarthritis. Journal of Medical Engineering and Technology, 28, 223–226.CrossRefGoogle Scholar
  7. 7.
    Li, L. P., Korhonen R. K., Iivarinen, J., Jurvelin, J. S., & Herzog, W. (2008). Fluid pressure driven fibril reinforcement in creep and relaxation tests of articular cartilage. Medical Engineering and Physics, 22, 182–189.Google Scholar
  8. 8.
    Huang, C. Y., Mow, V. C., & Ateshian, G. A. (2001). The role of flow-independent viscoelasticity in the biphasic tensile and compressive responses of articular cartilage. Journal of Biomechanical Engineering, 123, 410–417.CrossRefGoogle Scholar
  9. 9.
    Wu, J. Z., Herzog, W., & Epstein, M. (2000). Joint contact mechanics in the early stages of osteoarthritis. Medical Engineering and Physics, 22, 1–12.CrossRefGoogle Scholar
  10. 10.
    Carter, D. R., & Wong, M. (2003). Modelling cartilage mechanobiology. Philosophical Transactions of the Royal Society of London Series B, 358, 1461–1471.CrossRefGoogle Scholar
  11. 11.
    van Donkelaar, C. C., & Huiskes, R. (2006). The PTHrP-Ihh feedback loop in the embryonic growth plate allows PTHrP to control hypertrophy and Ihh to regulate proliferation. Biomechanics and Modeling in Mechanobiology, 6(1–2), 55–62.Google Scholar
  12. 12.
    Lacroix, D., & Prendergast, P. J. (2002). A mechano-regulation model for tissue differentiation during fracture healing: Analysis of gap size and loading. Journal of Biomechanics, 35, 1163–1171.CrossRefGoogle Scholar
  13. 13.
    Jelly, K. D., & Prendergast, P. J. (2006). Prediction of the optimal mechanical properties for a scaffold used in osteochondral defect repair. Tissue Engineering, 12, 2509–2529.CrossRefGoogle Scholar
  14. 14.
    Boykov, Y., & Funka-Lea, G. (2006). Graph cuts and efficient N-D image segmentation. International Journal of Computer Vision, 70(2), 109–131.CrossRefGoogle Scholar
  15. 15.
    Zhang, J., Zheng, J., & Cai, J. (2010). A diffusion approach to seeded image segmentation. In IEEE Computer Vision and Pattern Recognition (CVPR), San Francisco, USA (pp. 2125–2132).Google Scholar
  16. 16.
    Cremers, D., Rousson, M., & Deriche, R. (2007). A Review of statistical approaches to level set segmentation: Integrating color, texture, motion and shape. International Journal of Computer Vision, 72(2), 195–215.CrossRefGoogle Scholar
  17. 17.
    Nguyen, A., Cai, J., Zhang, J., & Zheng, J. (2012). Robust interactive image segmentation using convex active contours. IEEE Transactions on Image Processing, 21(8), 3734–3743.CrossRefMathSciNetGoogle Scholar
  18. 18.
    Chiang, P., Cai, Y. Y., Mak, K., & Zheng, J. M. (2013). A B-spline approach to phase unwrapping in tagged cardiac MRI for motion tracking. Magnetic Resonance in Medicine, 69, 1297–1309.Google Scholar
  19. 19.
    Sharma, N., & Aggarwal, L. M. (2010). Automated medical image segmentation techniques. Journal of Medical Physics, 35(1), 3–14.CrossRefGoogle Scholar
  20. 20.
    Gilles, B., & Magnenat-Thalmann, N. (2010). Musculoskeletal MRI segmentation using multi-resolution simplex meshes with medial representations. Medical Image Analysis, 14(3), 291–302.CrossRefGoogle Scholar
  21. 21.
    Schmid, J., Guitián, J., Gobbetti, E., & Magnenat-Thalmann, N. (2011). A GPU framework for parallel segmentation of volumetric images using discrete deformable models. The Visual Computer, 27(2), 85–95.CrossRefGoogle Scholar
  22. 22.
    Schmid, J., & Magnenat-Thalmann, N. (2008). MRI bone segmentation using deformable models and shape priors. Medical Image Computing and Computer-Assisted Intervention, 1, 119–126.Google Scholar
  23. 23.
    Fritscher, K. D., Grünerb, A., & Schubert, R. (2007). 3D image segmentation using combined shape-intensity prior models. International Journal of Computer Assisted Radiology and Surgery, 1, 341–350.CrossRefGoogle Scholar
  24. 24.
    Pereira, C. S., Alexandre, L. A., Mendonça, A. M., & Campilho, A. C. A. (2006). Multiclassifier approach for lung nodule classification. International Conference on Image Analysis and Recognition, 2, 612–623.CrossRefGoogle Scholar
  25. 25.
    Pohl, K. M., Fisher, J., Grimson, W. E. L., Kikinis, R., & Wells, W. M. (2006). A Bayesian model for joint segmentation and registration. NeuroImage, 31, 228–239.CrossRefGoogle Scholar
  26. 26.
    Brown, L. G. (1992). A survey of image registration techniques. ACM Computing Surveys, 24(4), 325–376.CrossRefGoogle Scholar
  27. 27.
    Johnson, H. J., & Christensen, G. E. (2002). Consistent landmark and intensity-based image registration. IEEE Transactions on Medical Imaging, 21(5), 450–461.CrossRefGoogle Scholar
  28. 28.
    Ding, L., Goshtasby, A., & Satter, M. (2001). Volume image registration by template matching. Image and Vision Computing, 19(12), 821–832.CrossRefGoogle Scholar
  29. 29.
    Zöllei, L., Grimson, E., Norbash, A., & Wells, W. (2001). 2D–3D Rigid registration of X-ray fluoroscopy and CT images using mutual information and sparsely sampled histogram estimators. In Proceedings of IEEE Computer Vision and Pattern Recognition (Vol. 2, pp. II-696–II-703).Google Scholar
  30. 30.
    McInerney, T., & Terzopoulos, D. (1996). Deformable models in biomedical images. Medical Image Analysis, 1(2), 91–108.CrossRefGoogle Scholar
  31. 31.
    Montagnat, J., & Delingette, H. (2001). A review of deformable surfaces: Topology, geometry and deformation. Image and Vision Computing, 19(14), 1023–1040.CrossRefGoogle Scholar
  32. 32.
    Terzopoulos, D., Witkin, A., & Kass, M. (1988). Constraints on deformable models: Recovering 3D shape and nonrigid motion. Artificial Intelligence, 36(1), 91–123.CrossRefMATHGoogle Scholar
  33. 33.
    Staib, L., & Duncan, J. (1992). Deformable fourier models for surface finding in 3D images. Visualization in Biomedical Computing, 1808, 90–104.Google Scholar
  34. 34.
    Malladi, R., Sethian, J. A., & Vemuri, B. C. (1995). Shape modeling with front propagation: A level set approach. IEEE Transactions on Pattern Analysis and Machine Intelligence, 17(2), 158–174.CrossRefGoogle Scholar
  35. 35.
    Lötjönen, J., & Mäkelä, T. (2001). Segmentation of MR images using deformable models: Application to cardiac images. International Journal of Bioelectromagnetism, 3(2), 37–45.Google Scholar
  36. 36.
    Delingette, H. (1999). General object reconstruction based on simplex meshes. International Journal of Computer Vision, 32(2), 111–146.CrossRefGoogle Scholar
  37. 37.
    Montagnat, J., & Delingette, H. (2005). 4D deformable models with temporal constraints: Application to 4D cardiac image segmentation. Medical Image Analysis, 9(1), 87–100.CrossRefGoogle Scholar
  38. 38.
    Gilles, B., Moccozet, L., & Magnenat-Thalmann, N. (2006). Anatomical modelling of the musculoskeletal system from MRI. In MICCAI’06 (Vol. 4190, pp. 289–296).Google Scholar
  39. 39.
    Gilles, B., Perrin, R., Magnenat-Thalmann, N., & Vallée, J.-P. (2005). Bones motion analysis from dynamic MRI: Acquisition and tracking. Academic Radiology, 12(10), 2385–2392.CrossRefGoogle Scholar
  40. 40.
    Montagnat, J., & Delingette, H. (2000). Space and time shape constrained deformable surfaces for 4D medical image segmentation. In Proceedings of Medical image Computing and Computer-Assisted Intervention (MICCAI ’00). Lecture Notes on Computer Science (Vol. 1935, pp. 196–205).Google Scholar
  41. 41.
    Wang, Y., & Staib, L. H. (2000). Physical model-based non-rigid registration incorporating statistical shape information. Medical Image Analysis, 4, 7–20.CrossRefGoogle Scholar
  42. 42.
    Liu, J., Huang, S., & Nowinski, W. L. (2008). A hybrid approach for segmentation of anatomic structures in medical images. International Journal of Computer Assisted Radiology and Surgery, 3(3/4), 213–219.CrossRefGoogle Scholar
  43. 43.
    Ng, H. P., Ong, S. H., Liu, J., Huang, S., Foong, K. W. C., Goh, P. S., et al. (2009). 3D segmentation and quantification of a masticatory muscle from MR data using patient-specific models and matching distributions. Journal of Digital Imaging, 22(5), 449–462.CrossRefGoogle Scholar
  44. 44.
    Guan, Y. Q., Cai, Y. Y., Lee, Y. T., & Opas, M. (2006). An Automatic method for identifying appropriate gradient magnitude for 3D boundary detection of confocal image stacks. Journal of Microscopy, 223(1), 66–72.CrossRefMathSciNetGoogle Scholar
  45. 45.
    Indhumathi, C., Cai, Y. Y., Guan, Y. Q., & Opas, M. (2009). 3D boundary extraction of confocal cellular images using higher order statistics. Journal of Microscopy, 235(2), 209–220.CrossRefMathSciNetGoogle Scholar
  46. 46.
    Friese, K. I., Blanke, P., & Wolter, F.-E. (2011). YaDiV—An open platform for 3D visualization and 3D segmentation of medical data. The Visual Computer, 27(3), 129–139.CrossRefGoogle Scholar
  47. 47.
    Chui, C. K., Chng, C. B., & Lau, D. P. C. (2011). Parallel processing for object oriented robotic simulation of tracheal-oesophageal puncture. In IEEE/SICE International Symposium on System Integration (SI International 2011), Kyoto, Japan (pp. 144–149).Google Scholar
  48. 48.
    Revost, P. (2011). Dynamic mechanical response of brain tissue in indentation in vivo, in situ and in vitro. Acta Biomateriala, 7, 4090–4101.CrossRefGoogle Scholar
  49. 49.
    Qiu, T. X., Teo, E. C., Yan, Y. B., & Lei, W. (2011). Finite element modelling of 3D coupled foot-boot model. Medical Engineering and Physics, 33(10), 1228–1233.CrossRefGoogle Scholar
  50. 50.
    Chui, C. K., et al. (2009). A component oriented software toolkit for patient-specific finite element model generation. Advances in Engineering Software, 40, 184–192.CrossRefMATHGoogle Scholar
  51. 51.
    Ahn, B., & Kim, J. (2010). Measurement and characterization of soft tissue behavior with surface deformation and force response under large deformations. Medical Image Analysis, 14, 138–148.CrossRefGoogle Scholar
  52. 52.
    Hui, J., Teo, Y. H., Li, L., & Lee, E. H. (2005). A comparative study of efficacy of bone marrow, periosteum and fat as source of MSC (mesenchymal stem cells) transfers in the treatment of partial growth arrest. Tissue Engineering, 11(5–6), 904–912.CrossRefGoogle Scholar
  53. 53.
    Thevendran, G., Sarraf, K. M., & Rosenfeld, P. R. (2012). Adult ankle fractures: Acute assessment and management. British Journal of Hospital Medicine, 31(5), 71–74.Google Scholar
  54. 54.
    Sandholm, A., Schwartz, C., Pronost, N., de Zee, M., Voigt, M., & Thalmann, D. (2011). Evaluation of a geometry-based knee joint compared to a planar knee joint. The Visual Computer, 27(2), 161–171.CrossRefGoogle Scholar
  55. 55.
    Kong, P. W., & van Haselen, J. (2010). Revisiting the influence of hip and knee angles on quadriceps excitation measured by surface electromyography. International Sport Medical Journal, 11(2), 313–323.Google Scholar
  56. 56.
    King, M. A., Kong, P. W., & Yeadon, M. R. (2009). Determining effective subject-specific strength levels for forward dives using computer simulations of recorded performances. Journal of Biomechanics, 42(16), 2672–2677.CrossRefGoogle Scholar
  57. 57.
    Loh, Y. J., Tjan, S. Y., Xu, D., Thia, E., & Kong, K. H. (2010). A feasibility study using interactive commercial off-the-shelf computer gaming in upper limb rehabilitation in patients after stroke. Journal of Rehabilitation Medicine, 42(5), 437–441.CrossRefGoogle Scholar
  58. 58.
    Huber, M., Rabin, B., Docan, C., Burdea, G., Abdelbaky, M., & Golomb, M. (2010). Feasibility of modified remotely-monitored in-home gaming technology for improving hand function in adolescents with cerebral palsy. IEEE Transactions on Information Technology in Biomedical Engineering, 14(2), 526–534.CrossRefGoogle Scholar
  59. 59.
    Burdea, G., Cioi, D., Martin, J., Fensterheim, D., & Holenski, M. (2010). The Rutgers arm II rehabilitation system—a feasibility study. IEEE Transactions on Neural Systems and Rehabilitaion Engineering, 18(5), 505–514.CrossRefGoogle Scholar
  60. 60.
    Arbabi, E., Boulic, R., & Thalmann, D. (2009). Fast collision detection methods for joint surfaces. Journal of Biomechanics, 42(2), 91–99.CrossRefGoogle Scholar
  61. 61.
    Teschner, M., et al. (2004). Collision detection for deformable objects. In Proceedings of Eurographics ’04 State-of-the-Art-Reports ( pp. 119–139).Google Scholar
  62. 62.
    Han, S.-K., Federico, S., Epstein, M., & Herzog, W. (2005). An articular cartilage contact model based on real surface geometry. Journal of Biomechanics, 38(1), 179–184.CrossRefGoogle Scholar
  63. 63.
    DeFrate, L. E., Sun, H., Gill, T. J., Rubash, H. E., & Li, G. (2004). In vivo tibiofemoral contact analysis using 3D MRI-based knee models. Journal of Biomechanics, 37(10), 1499–1504.CrossRefGoogle Scholar
  64. 64.
    Shi, Q., Hashizume, H., Inoue, H., Miyake, T., & Nagayama, N. (1995). Finite element analysis of pathogenesis of osteoarthritis in the first carpometacarpal joint. Acta Medica Okayama, 49(1), 43–51.Google Scholar
  65. 65.
    Armand, M., et al. (2004). Computer-aided orthopaedic surgery with near-real-time biomechanical feedback. Johns Hopkins APL Technical Digest, 25(3), 242–252.Google Scholar
  66. 66.
    Harman, M. K., Banks, S. A., Fregly, B. J., Sawyer, W. G., & Hodge, W. A. (2005). Biomechanical mechanisms for damage: Retrieval analysis and computational wear predictions in total knee replacements. Journal of Mechanics in Medicine and Biology, 5(3), 469–475.CrossRefGoogle Scholar
  67. 67.
    Lin, M. C., & Canny, J. F. (1991). A fast algorithm for incremental distance calculation. In Proceedings of IEEE International Conference on Robotics and Automation (pp. 1008–1014).Google Scholar
  68. 68.
    Larsson, T., & Akenine-Möller, T. (2001). Collision detection for continuously deforming bodies. In Proceedings of Eurographics.Google Scholar
  69. 69.
    Larsson, T., & Akenine-Möller, T. (2003). Efficient collision detection for models deformed by morphing. The Visual Computer, 19(2–3), 164–174.MATHGoogle Scholar
  70. 70.
    Maciel, A., Boulic, R., & Thalmann, D. (2007). Efficient collision detection within deforming spherical sliding contact. IEEE Transactions in Visualization and Computer Graphics, 13(3), 518–529.CrossRefGoogle Scholar
  71. 71.
    Kettelkamp, D. B., Wenger, D. R., Chao, E. Y. S., & Thompson, C. (1976). Results of proximal tibial osteotomy. The Journal of Bone and Joint Surgery, 58-A(7), 952–960.Google Scholar
  72. 72.
    Waugh, W. (1976). Tibial osteotomy in the management of ostéoarthritis of the knee. Clinical Orthopaedics and Related Research, 210, 56–61.Google Scholar
  73. 73.
    Maquet, P. (1976). Biomécanique du genou (p. 237). Berlin: Springer.Google Scholar
  74. 74.
    Vainionpaa, S., Laike, E., Kirves, P., & Tiusanen, P. (1981). Tibial osteotomy for osteo-arthritis of the knee (a five to ten year follow-up study). The Journal of Bone and Joint Surgery, 63(A-6), 398–945.Google Scholar
  75. 75.
    Coventry, M. B. (1982). Long term results of upper tibial osteotomy for degenerative arthritis of the knee. Acta Orthopaedica Belgica, 48(1), 139–156.Google Scholar
  76. 76.
    Hernigou, P., Medevielle, D., Debeyre, J., & Goutallier, D. (1987). Proximal tibial osteotomy for osteo-arthritis with varus deformity. The Journal of Bone and Joint Surgery, 69A, 332–354.Google Scholar
  77. 77.
    Thomine, J. M. (1989). Les ostéotomies dans la gonarthrose fémoro-tibiale latéralisée. Théorie et pratique, in Cahier d’enseignement de la SOFCOT \(N^{\circ }\) 34 (pp. 99–112). Paris, France.Google Scholar
  78. 78.
    Arbabi, E., Chegini, S., Boulic, R., Tannast, M., Ferguson, S. J., & Thalmann, D. (2010). The penetration depth method—a novel real time strategy for evaluating femoro-acetabular impingement. Journal of Orthopaedic Research, 28(7), 880–886.Google Scholar
  79. 79.
    Maurel, W., & Thalmann, D. (1999). A case study analysis on human upper limb modeling for dynamic simulation. Computer Methods in Biomechanics and Biomechanical Engineering, 1, 65–82.CrossRefGoogle Scholar
  80. 80.
    Boulic, R., Magnenat-Thalmann, N., & Thalmann, D. (1990). A global human walking model with real time kinematic personification. The Visual Computer, 6(6), 344–358.CrossRefGoogle Scholar
  81. 81.
    Glardon, P., Boulic, R., & Thalmann, D. (2006). Robust on-line adaptive footplant detection and enforcement for locomotion. The Visual Computer, 22(3), 194–209.CrossRefGoogle Scholar
  82. 82.
    Carvalho, S. R., Boulic, R., & Thalmann, D. (2007). Interactive low-dimensional human motion synthesis by combining motion models and PIK. Computer Animation and Virtual, 18(4–5), 493–503.CrossRefGoogle Scholar
  83. 83.
    Thirion, J. P. (1995). Fast non-rigid matching of 3D medical images. Technical Report 2547. Marseilles: INRIA.Google Scholar
  84. 84.
    Charbonnier, C., Gilles, B., & Magnenat-Thalmann, N. (2007). A semantic-driven clinical examination platform. In Surgetica’2007, Computer-Aided Medical Interventions: Tools and Applications.Google Scholar
  85. 85.
    Chiang, P., et al. (2012). A VR simulator for intra-cardiac interventional procedure: Concept, design and implementation. IEEE Computer Graphics and Applications, 33(1), 44–57.Google Scholar
  86. 86.
    Pan, J., Chang, J., Yang, X., Zhang, J. J., et al. (2011). A medical VR simulator in laparoscopic rectum surgery. Cyber Theraphy and Rehabilitation, 4(4), 19–20.Google Scholar
  87. 87.
    Cai, Y. Y., Chia, N., Thalmann, D., Kee, N., Zheng, J., & Thalmann, N. Design and development of a virtual dolphinarium for children with autism. IEEE Transaction on Neural System and Rehabilitation Engineering (to appear).Google Scholar
  88. 88.
    Chiang, P., Zheng, J. M., Mak, K. H., Thalmann, N., & Cai, Y. Y. (2012). Progressive surface reconstruction for heart mapping procedure. Computer-Aided Design, 44, 289–299.CrossRefGoogle Scholar
  89. 89.
    Cai, Y. Y., Zheng, J. M., Chiang, P., Thalmann, N., & Mak, K. H. (2012). Method of progressive and real-time intra-cardiac surface reconstruction, US Patent Filed.Google Scholar
  90. 90.
    Chong, W. H., Goh, W., Tang, H. N., Chan, W. P., & Choo, S. (2012). Service practice evaluation of the early intervention programs for infants and young children in Singapore. Children’s Health Care, 41(4), 281–301.Google Scholar
  91. 91.
    Chin, P. L., et al. (2011). Intraoperative morphometric study of gender differences in Asian femurs. Journal of Arthroplasty, 26(7), 984–988.Google Scholar

Copyright information

© Springer-Verlag London 2014

Authors and Affiliations

  • Nadia Magnenat Thalmann
    • 1
    • 2
  • Hon Fai Choi
    • 1
  • Daniel Thalmann
    • 2
    • 3
  1. 1.MIRALabUniversity of GenevaGenevaSwitzerland
  2. 2.Institute for Media InnovationNanyang Technological UniversitySingaporeSingapore
  3. 3.EPFLLausanneSwitzerland

Personalised recommendations