Abstract
During a medical examination, clinicians build a health record containing all available information about their patient. A promising way to support their decisions is to retrieve similar patient records from a medical archive. Confronted to similar cases, clinicians may confirm or revise their decisions by analogy reasoning. In order to retrieve patient records, two challenges need to be addressed. First, how to characterize complex elements in patient records (images, videos, etc.)? Second, how to combine heterogeneous elements in these records (demographic and clinical data, images, videos, etc.) in order to define clinically-relevant similarity metrics? After a short review of content-based image, video or health record retrieval techniques, this chapter presents the solutions we have developed for two applications in ophthalmology: computer-aided retinal diagnosis and computer-aided eye surgery. Medical archives are a great asset to develop the medical decision supports of tomorrow. Thanks to major advances in information retrieval, network data storage (cloud), with related topics such as security, virtually any medical decision problem can benefit from information stored in medical archives.
Keywords
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Abouelenien, M., Wan, Y., & Saudagar, A. (2012). Feature and decision level fusion for action recognition. In Proceedings of International Conference on Computing, Communications and Networking Technologies (ICCCNT) (pp. 1–7).
Amores, J. (2013). Multiple instance classification: Review, taxonomy and comparative study. Artificial Intelligence, 201, 81–105.
André, B., Vercauteren, T., Buchner, A. M., Shahid, M. W., Wallace, M. B., & Ayache, N. (2010). An image retrieval approach to setup difficulty levels in training systems for endomicroscopy diagnosis. In Proceedings of Medical Image Computing and Computer Assisted Interventions (MICCAI) (pp. 480–487).
André, B., Vercauteren, T., Buchner, A. M., Wallace, M. B., & Ayache, N. (2012). Learning semantic and visual similarity for endomicroscopy video retrieval. IEEE Transactions on Medical Imaging, 31(6), 1276–1288.
André, B., Vercauteren, T., Wallace, M. B., Buchner, A. M., & Ayache, N. (2010). Endomicroscopic video retrieval using mosaicing and visual words. In Proceedings of IEEE International Symposium on Biomedical Imaging (ISBI) (pp. 1419–1422).
Avni, U., Greenspan, H., Konen, E., Sharon, M., Goldberger, J. (2011). X-ray categorization and retrieval on the organ and pathology level, using patch-based visual words. IEEE Transactions on Medical Imaging, 30(3), 733–746.
Bettadapura, V., Schindler, G., Ploetz, T., & Essa, I. (2013). Augmenting bag-of-words: Data-driven discovery of temporal and structural information for activity recognition. In Proceedings of IEEE Computer Vision and Pattern Recognition (CVPR) (pp. 2619–2626).
Bichindaritz, I. (2006). Mémoire: A framework for semantic interoperability of case-based reasoning systems in biology and medicine. Artificial Intelligence in Medicine, 36(2), 177–192.
Bichindaritz, I., & Marling, C. (2006). Case-based reasoning in the health sciences: What’s next? Artificial Intelligence in Medicine, 36(2), 127–135.
Blum, T., Feussner, H., & Navab, N. (2010). Modeling and segmentation of surgical workflow from laparoscopic video. In Proceedings of Medical Image Computing and Computer Assisted Interventions (MICCAI) (pp. 400–407).
Bruno, E., Moenne-Loccoz, N., & Marchand-Maillet, S. (2008). Design of multimodal dissimilarity spaces for retrieval of video documents. IEEE Transactions on Pattern Analysis and Machine Intelligence, 30(9), 1520–1533.
Cauvin, J. M., Le Guillou, C., Solaiman, B., Robaszkiewicz, M., Le Beux, P., & Roux, C. (2003). Computer-assisted diagnosis system in digestive endoscopy. IEEE Transactions on Information Technology, 7(4), 256–262.
Chatzichristofis, S. A., Iakovidou, C., Boutalis, Y., Marques, O. (2013). Co.Vi.Wo.: Color visual words based on non-predefined size codebooks. IEEE Transactions on Cybernetics, 43(1), 192–205.
Decencière, E., Cazuguel, G., Zhang, X., et al. (2013). TeleOphta: Machine learning and image processing methods for teleophthalmology. IRBM, 34(2), 196–203.
Douze, M., Jégou, H., Schmid, C. (2010). An image-based approach to video copy detection with spatio-temporal post-filtering. IEEE Transactions on Multimedia, 12(4), 257–266.
Droueche, Z., Lamard, M., Cazuguel, G., Quellec, G., Roux, C., & Cochener, B. (2011). Content-based medical video retrieval based on region motion trajectories. In Proceedings of International Federation for Medical and Biological Engineering (IFMBE) (pp. 622–625).
Droueche, Z., Lamard, M., Cazuguel, G., Quellec, G., Roux, C., & Cochener, B. (2012). Motion-based video retrieval with application to computer-assisted retinal surgery. In Proceedings of IEEE Engineering in Medicine and Biology Society (EMBS) (pp. 4962–4965).
Dyana, A., Subramanian, M. P., & Das, S. (2009). Combining features for shape and motion trajectory of video objects for efficient content based video retrieval. In Proceedings of International Conference on Advances in Pattern Recognition (ICAPR) (pp. 113–116).
Gao, H. P., & Yang, Z. Q. (2010). Content based video retrieval using spatiotemporal salient objects. In Proceedings of International Petroleum Technology Conference (IPTC) (pp. 689–692).
Haro, B. B., Zappella, L., & Vidal, R. (2012). Surgical gesture classification from video data. In Proceedings of Medical Image Computing and Computer Assisted Interventions (MICCAI) (pp. 34–41).
Haux, R. (2006). Health information system: Past, present, and future. International Journal of Medical Informatics, 75(3–4), 268–281.
Hoi, S. C. H., & Lyu, M. R. (2007). A multimodal and multilevel ranking framework for content-based video retrieval. In Proceedings of International Conference on Acoustics, Speech and Signal Processing (ICASSP) (pp. 1225–1228)
Hu, W., Xie, D., Fu, Z., Zeng, W., & Maybank, S. (2007). Semantic-based surveillance video retrieval. IEEE Transactions on Image Processing, 16(4), 1168–1181.
Ji, R., Duan, L. Y., Chen, J., Xie, L., Yao, H., & Gao, W. (2013). Learning to distribute vocabulary indexing for scalable visual search. IEEE Transactions on Multimedia, 15(1), 153–166.
Juan, K., & Cuiying, H. (2010). Content-based video retrieval system research. In Proceedings of International Conference on Computer Science and Information Technology (ICCSIT) (pp. 701–704).
Lalys, F., Riffaud, L., Bouget, D., & Jannin, P. (2012). A framework for the recognition of high-level surgical tasks from video images for cataract surgeries. IEEE Transactions on Biomedical Engineering, 59(4), 966–976.
Liu, Z., Li, H., Zhou, W., Zhao, R., & Tian, Q. (2014). Contextual hashing for large-scale image search. IEEE Transactions on Image Processing, 23(4), 1606–1614.
Mansencal, B., Benois-Pineau, J., Vieux, R., & Domenger, J. (2012). Search of objects of interest in videos. In Proceedings of Content-Based Multimedia Indexing (CBMI) (pp. 1–6).
Müller, H., Michoux, N., Bandon, D., & Geissbuhler, A. (2004). A review of content-based image retrieval systems in medical applications—clinical benefits and future directions. International Journal of Medical Informatics, 73(1), 1–23.
Müller, H., Seco de Herrera, A. G., Kalpathy-Cramer, J., Fushman, D. D., Antani, S., & Eggel, I. (2012). Overview of the ImageCLEF 2012 medical image retrieval and classification tasks. In Conference and Labs of the Evaluation Forum (CLEF) 2012 working notes.
Naturel, X., & Gros, P. (2008). Detecting repeats for video structuring. Multimedia Tools and Applications, 38(2), 233–252.
Niemeijer, M., van Ginneken, B., Cree, M. J., et al. (2010). Retinopathy online challenge: Automatic detection of microaneurysms in digital color fundus photographs. IEEE Transactions on Medical Imaging, 29(1), 185–195.
Pan, W., Coatrieux, G., Cuppens, N., Cuppens, F., & Roux, C. (2010). An additive and lossless watermarking method based on invariant image approximation and haar wavelet transform. In Proceedings of IEEE Engineering in Medicine and Biology Society (EMBS) (pp. 4740–4743).
Patel, B. V., Deorankar, A. V., & Meshram, B. B. (2010). Content based video retrieval using entropy, edge detection, black and white color features. In Proceedings of International Conference on Chemical Engineering and Technology (ICCET) (pp. 272–276).
Perner, P. (Ed.). (2008). Case-based reasoning on images and signals. Studies in Computational Intelligence (Vol. 73). Heidelberg: Springer.
Pires, R., Jelinek, H. F., Wainer, J., Goldenstein, S., Valle, E., & Rocha, A. (2013). Assessing the need for referral in automatic diabetic retinopathy detection. IEEE Transactions on Biomedical Engineering, 60(12), 3391–3398.
Quantin, C., Cohen, O., Riandey, B., & Allaert, F. A. (2007). Unique patient concept: A key choice for european epidemiology. International Journal of Medical Informatics, 76(5–6), 419–426.
Quellec, G., Charrière, K., Lamard, M., Droueche, Z., Roux, C., & Cochener, B. (2014). Real-time recognition of surgical tasks in eye surgery videos. Medical Image Analysis, 18(3), 579–590.
Quellec, G., Lamard, M., Abràmoff, M. D., Decencière, E., Lay, B., & Erginay, A. (2012). A multiple-instance learning framework for diabetic retinopathy screening. Medical Image Analysis, 16(6), 1228–1240.
Quellec, G., Lamard, M., Bekri, L., Cazuguel, G., Roux, C., & Cochener, B. (2010). Medical case retrieval from a committee of decision trees. IEEE Transactions on Information Technology in Biomedicine,14(5), 1227–1235.
Quellec, G., Lamard, M., Cazuguel, G., Bekri, L., Daccache, W., & Roux, C. (2011). Automated assessment of diabetic retinopathy severity using content-based image retrieval in multimodal fundus photographs. Investigative Ophthalmology and Visual Science, 52(11), 8342–8348.
Quellec, G., Lamard, M., Cazuguel, G., Cochener, B., & Roux, C. (2009). Multimodal information retrieval based on DSmT. Application to computer aided medical diagnosis. In F. Smarandache & J. Dezert (Eds.), Advances and applications of DSmT for information fusion III, chap. 18 (pp. 471–502). Ann Harbor: American Research Press.
Quellec, G., Lamard, M., Cazuguel, G., Cochener, B., & Roux, C. (2010). Adaptive nonseparable wavelet transform via lifting and its application to content-based image retrieval. IEEE Transactions on Image Processing, 19(1), 25–35.
Quellec, G., Lamard, M., Cazuguel, G., Cochener, B., & Roux, C. (2010). Wavelet optimization for content-based image retrieval in medical databases. Medical Image Analysis, 14(2), 227–241.
Quellec, G., Lamard, M., Cazuguel, G., Cochener, B., & Roux, C. (2012). Fast wavelet-based image characterization for highly adaptive image retrieval. IEEE Transactions on Image Processing, 21(4), 1613–1623.
Quellec, G., Lamard, M., Cazuguel, G., Roux, C., & Cochener, B. (2008). Multimodal medical case retrieval using dezert-smarandache theory with a priori knowledge. In Proceedings of International Federation for Medical and Biological Engineering (IFMBE) (pp. 716–719).
Quellec, G., Lamard, M., Cazuguel, G., Roux, C., & Cochener, B. (2011). Case retrieval in medical databases by fusing heterogeneous information. IEEE Transactions on Medical Imaging, 30(1), 108–118.
Quellec, G., Lamard, M., Cochener, B., & Cazuguel, G. (2014). Real-time segmentation and recognition of surgical tasks in cataract surgery videos. IEEE Trans Med Imaging, 33(12), 2352–2360.
Quellec, G., Lamard, M., Cochener, B., Droueche, Z., Lay, B., & Chabouis, A. et al. (2012). Studying disagreements among retinal experts through image analysis. In Proceedings of IEEE Engineering in Medicine and Biology Society (EMBS) (pp. 5959–5962).
Quellec, G., Lamard, M., Cochener, B., Roux, C., & Cazuguel, G. (2012). Comprehensive wavelet-based image characterization for content-based image retrieval. In Proceedings of the Conference on Content-Based Multimedia Indexing (CBMI).
Quellec, G., Lamard, M., Droueche, Z., Cochener, B., Roux, C., & Cazuguel, G. (2013). A polynomial model of surgical gestures for real-time retrieval of surgery videos. In Lecture Notes in Computer Science: Vol. 7723. Proceedings MCBR-CDS (pp. 10–20).
Ren, R., & Collomosse, J. (2012). Visual sentences for pose retrieval over low-resolution cross-media dance collections. IEEE Transactions on Multimedia, 14(6), 1652–1661.
Rublee, E., Rabaud, V., Konolige, K., & Bradski, G. R. (2011). ORB: An efficient alternative to SIFT or SURF. In Proceedings of IEEE International Conference on Computer Vision (ICCV) (pp. 2564–2571).
Safran, C., Bloomrosen, M., Hammond, W. E., et al. (2007). Toward a national framework for the secondary use of health data: An american medical informatics association white paper. Journal of the American Medical Informatics Association, 14(1), 1–9.
Sivic, J., Russell, B. C., Efros, A. A., Zisserman, A., & Freeman, W. T. (2005). Discovering objects and their location in images. In Proceedings of IEEE International Conference on Computer Vision (ICCV) (pp. 370–377).
Strat, S. T., Benoit, A., & Lambert, P. (2013). Retina enhanced SIFT descriptors for video indexing. In Proceedings of the Conference on Content-Based Multimedia Indexing (CBMI) (pp. 201–206).
Sweldens, W. (1998). The lifting scheme: A construction of second generation wavelets. SIAM Journal on Mathematical Analysis, 29(2), 511–546.
Syeda-Mahmood, T., Ponceleon, D., & Yang, J. (2005). Validating cardiac echo diagnosis through video similarity. In Proceedings of ACM Multimedia (pp. 527–530).
Tao, L., Elhamifar, E., Khudanpur, S., Hager, G. D., & Vidal, R. (2012). Sparse hidden markov models for surgical gesture classification and skill evaluation. In Proceedings of Information Processing in Computer-Assisted Interventions (IPCAI) (pp. 167–177).
Tao, L., Zappella, L., Hager, G. D., & Vidal, R. (2013). Surgical gesture segmentation and recognition. In Lecture Notes in Computer Science: Vol. 8151 (pp. 339–46).
Tsikrika, T., Kludas, J., & Popescu, A. (2012). Building reliable and reusable test collections for image retrieval: The Wikipedia task at ImageCLEF. IEEE Multimedia, 19(3), 24–33.
Tutac, A. E., Cretu, V. I., & Racoceanu, D. (2010). Spatial representation and reasoning in breast cancer grading ontology. In Proceedings of International Joint Conference on Computational Cybernetics and Technical Informatics (ICCC-CONTI) (pp. 89–94).
Vieux, R., Benois-Pineau, J., Domenger, J. P. (2012). Content based image retrieval using bag of regions. In Proceedings of Multimedia Modeling (MMM) (pp. 507–517).
Xu, D., & Chang, S. F. (2008). Video event recognition using kernel methods with multilevel temporal alignment. IEEE Transactions on Pattern Analysis and Machine Intelligence, 30(11), 1985–1997.
Yang, Y., & Newsam, S. (2013). Geographic image retrieval using local invariant features. IEEE Transactions on Geoscience Remote Sensing, 51(2), 818–832.
Yuan, C., Li, X., Hu, W., Ling, H., & Maybank, S. J. (2014) Modeling geometric-temporal context with directional pyramid co-occurrence for action recognition. IEEE Transactions on Image Processing, 23(2), 658–672.
Zappella, L., Béjar, B., Hager, G., & Vidal, R. (2013). Surgical gesture classification from video and kinematic data. Medical Image Analysis, 17(7), 732–745.
Zheng, L., & Wang, S. (2013). Visual phraselet: refining spatial constraints for large scale image search. IEEE Signal Processing Letters, 20(4), 391–394.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2015 Springer International Publishing Switzerland
About this chapter
Cite this chapter
Quellec, G., Lamard, M., Cochener, B., Cazuguel, G. (2015). Multimedia Information Retrieval from Ophthalmic Digital Archives. In: Briassouli, A., Benois-Pineau, J., Hauptmann, A. (eds) Health Monitoring and Personalized Feedback using Multimedia Data. Springer, Cham. https://doi.org/10.1007/978-3-319-17963-6_6
Download citation
DOI: https://doi.org/10.1007/978-3-319-17963-6_6
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-17962-9
Online ISBN: 978-3-319-17963-6
eBook Packages: Computer ScienceComputer Science (R0)