Abstract
Nonlinear Laser Scanning Microscopy (NLSM) techniques have been demonstrated in the past two decades as powerful imaging tools for disease diagnostics (DD). Currently, in most DD related experiments the interpretation of NLSM data sets is performed by trained specialists. Such approaches are both time consuming and prone to errors due to inter- and intra-observer discrepancies. The Bag-of-Features (BoF) paradigm has demonstrated its potential usefulness with respect to automated data classification in the frame of multiple experiments, but its intersections with the field of NLSM are at this moment scarce, to say the least. In this paper we review recent progress on DD using NLSM, and discuss necessary steps and potential future perspectives for merging NLSM and BoF to achieve complex frameworks for automated DD with high sensitivity and specificity.
Similar content being viewed by others
References
Alahi, A., Ortiz, R., Vandergheynst, P.: Freak: fast retina keypoint. In: 2012 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 510–517. IEEE (2012)
Aly, M., Munich, M., Perona, P.: Multiple dictionaries for bag of Words Large Scale Image Search. In: IEEE International Conference on Image Processing (ICIP) (2011)
Aucouturier, J.-J., Defreville, B., Pachet, F.: The bag-of-frames approach to audio pattern recognition: a sufficient model for urban soundscapes but not for polyphonic music. J. Acoust. Soc. Am. 122(2), 881–891 (2007)
Baenke, F., Peck, B., Miess, H., Schulze, A.: Hooked on fat: the role of lipid synthesis in cancer metabolism and tumour development. Dis. Models Mech. 6(6), 1353–1363 (2013)
Bay, H., Ess, A., Tuytelaars, T., Van Gool, L.: Speeded-up robust features (SURF). Comput. Vis. Image Underst. 110(3), 346–359 (2008)
Birk, J.W., Tadros, M., Moezardalan, K., Nadyarnykh, O., Forouhar, F., Anderson, J., Campagnola, P.: Second harmonic generation imaging distinguishes both high-grade dysplasia and cancer from normal colonic mucosa. Dig. Dis. Sci. 59(7), 1529–1534 (2014)
Bonnans, C., Chou, J., Werb, Z.: Remodelling the extracellular matrix in development and disease. Nat. Rev. Mol. Cell Biol. 15(12), 786–801 (2014)
Brelstaff, G., Bicego, M., Culeddu, N., Chessa, M.: Bag of peaks: interpretation of nmr spectrometry. Bioinformatics 25(2), 258–264 (2009)
Breunig, H.G., Weinigel, M., Bückle, R., Kellner-Höfer, M., Lademann, J., Darvin, M.E., Sterry, W., König, K.: Clinical coherent anti-Stokes Raman scattering and multiphoton tomography of human skin with a femtosecond laser and photonic crystal fiber. Laser Phys. Lett. 10(2), 025604 (2013)
Caicedo, J.C., Cruz, A., Gonzalez, F.A.: Histopathology image classification using bag of features and kernel functions. In: Artificial Intelligence in Medicine. pp. 126–135. Springer (2009)
Cao, Y., Wang, C., Li, Z., Zhang, L., Zhang, L.: Spatial-bag-of-features. In: 2010 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3352–3359. IEEE (2010)
Chakraborty, S., Nian, F.-S., Tsai, J.-W., Karmenyan, A., Chiou, A.: Quantification of the metabolic state in cell-model of Parkinson’s disease by fluorescence lifetime imaging microscopy. Sci. Rep. 6, 19145 (2016)
Chen, X., Nadiarynkh, O., Plotnikov, S., Campagnola, P.J.: Second harmonic generation microscopy for quantitative analysis of collagen fibrillar structure. Nat. Protoc. 7(4), 654–669 (2012)
Cicchi, R., Sturiale, A., Nesi, G., Kapsokalyvas, D., Alemanno, G., Tonelli, F., Pavone, F.S.: Multiphoton morpho-functional imaging of healthy colon mucosa, adenomatous polyp and adenocarcinoma. Biomed. Opt. Express 4(7), 1204–1213 (2013)
Dalal, N., Triggs, B.: Histograms of oriented gradients for human detection. In: Computer Vision and Pattern Recognition, 2005. CVPR 2005. In: Conference on IEEE Computer Society, pp. 886–893. IEEE (2005)
Evans, C.L., Xu, X., Kesari, S., Xie, X.S., Wong, S.T., Young, G.S.: Chemically-selective imaging of brain structures with CARS microscopy. Opt. Express 15(19), 12076–12087 (2007)
Eweiwi, A., Cheema, M.S., Bauckhage, C.: Action recognition in still images by learning spatial interest regions from videos. Pattern Recogn. Lett. 51, 8–15 (2015)
Fu, Z., Lu, G., Ting, K.M., Zhang, D.: Music classification via the bag-of-features approach. Pattern Recogn. Lett. 32(14), 1768–1777 (2011)
Gauderon, R., Lukins, P., Sheppard, C.: Second-harmonic generation imaging. In: Optics and Lasers in Biomedicine and Culture. pp. 66–69. Springer (2000)
Giacomelli, M.G., Sheikine, Y., Vardeh, H., Connolly, J.L., Fujimoto, J.G.: Rapid imaging of surgical breast excisions using direct temporal sampling two photon fluorescent lifetime imaging. Biomed. Opt. Express 6(11), 4317–4325 (2015)
Gulledge, C., Dewhirst, M.: Tumor oxygenation: a matter of supply and demand. Anticancer Res. 16(2), 741–749 (1995)
Hamasha, K., Mohaidat, Q.I., Putnam, R.A., Woodman, R.C., Palchaudhuri, S., Rehse, S.J.: Sensitive and specific discrimination of pathogenic and nonpathogenic Escherichia coli using Raman spectroscopy—a comparison of two multivariate analysis techniques. Biomed. Opt. Express 4(4), 481–489 (2013)
Huang, X., Irmak, S., Lu, Y., Pipinos, I., Casale, G., Subbiah, J.: Spontaneous and coherent anti-Stokes Raman spectroscopy of human gastrocnemius muscle biopsies in CH-stretching region for discrimination of peripheral artery disease. Biomed. Opt. Express 6(8), 2766–2777 (2015)
Kantelhardt, S.R., Kalasauskas, D., König, K., Kim, E., Weinigel, M., Uchugonova, A., Giese, A.: In vivo multiphoton tomography and fluorescence lifetime imaging of human brain tumor tissue. J. Neuro-Oncol. 127(3), 473–482 (2016)
Kong, K., Kendall, C., Stone, N., Notingher, I.: Raman spectroscopy for medical diagnostics—From in vitro biofluid assays to in vivo cancer detection. Adv. Drug Deliv. Rev. 89, 121–134 (2015)
Krahmer, N., Farese, R.V., Walther, T.C.: Balancing the fat: lipid droplets and human disease. EMBO Mol. Med. 5(7), 973–983 (2013)
Lazebnik, S., Schmid, C., Ponce, J.: Beyond bags of features: Spatial pyramid matching for recognizing natural scene categories. In: 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 2169–2178. IEEE (2006)
Leutenegger, S., Chli, M., Siegwart, R.Y.: BRISK: Binary robust invariant scalable keypoints. In: 2011 IEEE International Conference on Computer Vision (ICCV), pp. 2548–2555. IEEE (2011)
Li, L., Chen, Z., Wang, X., Liu, X., Jiang, W., Zhuo, S., Guan, G., Chen, J.: Visualization of tumor response to neoadjuvant therapy for rectal carcinoma by nonlinear optical imaging. IEEE J. Sel. Top. Quantum Electron. 22(3), 6800206 (2016)
Li L, Li H, Chen Z, Zhuo S, Feng C, Yang Y, Guan G, Chen J Layer-resolved colorectal tissues using nonlinear microscopy. Lasers Med. Sci. 1–9 (2015)
Lowe, D.G.: Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vision 60(2), 91–110 (2004)
Lu, P., Weaver, V.M., Werb, Z.: The extracellular matrix: a dynamic niche in cancer progression. J. Cell Biol. 196(4), 395–406 (2012)
Makino, T., Jain, M., Montrose, D.C., Aggarwal, A., Sterling, J., Bosworth, B.P., Milsom, J.W., Robinson, B.D., Shevchuk, M.M., Kawaguchi, K.: Multiphoton tomographic imaging: a potential optical biopsy tool for detecting gastrointestinal inflammation and neoplasia. Cancer Prev. Res. 5(11), 1280–1290 (2012)
Nanni, L., Costa, Y.M., Lumini, A., Kim, M.Y., Baek, S.R.: Combining visual and acoustic features for music genre classification. Expert Syst. Appl. 45, 108–117 (2016)
Nanni, L., Melucci, M.: Combination of projectors, standard texture descriptors and bag of features for classifying images. Neurocomputing 173, 1602–1614 (2016)
Nordestgaard, B.G., Varbo, A.: Triglycerides and cardiovascular disease. Lancet 384(9943), 626–635 (2014)
O’Hara, S., Draper, B.A.: Introduction to the bag of features paradigm for image classification and retrieval. arXiv preprint arXiv:1101.3354 (2011)
Pezacki, J.P., Blake, J.A., Danielson, D.C., Kennedy, D.C., Lyn, R.K., Singaravelu, R.: Chemical contrast for imaging living systems: molecular vibrations drive CARS microscopy. Nat. Chem. Biol. 7(3), 137–145 (2011)
Raykar, V.C., Yu, S., Zhao, L.H., Jerebko, A., Florin, C., Valadez, G.H., Bogoni, L., Moy, L.: Supervised learning from multiple experts: whom to trust when everyone lies a bit. In: Proceedings of the 26th Annual international conference on machine learning, pp. 889–896. ACM (2009)
Rublee, E., Rabaud, V., Konolige, K., Bradski, G.: ORB: an efficient alternative to SIFT or SURF. In: 2011 IEEE International Conference on Computer Vision (ICCV), pp. 2564–2571. IEEE (2011)
Sadek, I., Sidibé, D., Meriaudeau, F.: Automatic discrimination of color retinal images using the bag of words approach. In: SPIE Medical Imaging 2015, pp. 94141 J-94141 J-94148. International Society for Optics and Photonics
Schie, I.W., Krafft, C., Popp, J.: Applications of coherent Raman scattering microscopies to clinical and biological studies. Analyst 140(12), 3897–3909 (2015)
So, P.T., Dong, C.Y., Masters, B.R., Berland, K.M.: Two-photon excitation fluorescence microscopy. Annu. Rev. Biomed. Eng. 2(1), 399–429 (2000)
Stanciu, S.G., Coltuc, D., Tranca, D.E., Stanciu, G.A.: Matching DSIFT descriptors extracted from CSLM images. Engineering 5(10), 199–202 (2013)
Stanciu, S.G., Hristu, R., Boriga, R., Stanciu, G.A.: On the suitability of SIFT technique to deal with image modifications specific to Confocal Scanning Laser Microscopy. Microsc. Microanal. 16(05), 515–530 (2010)
Stanciu, S.G., Hristu, R., Stanciu, G.A.: Influence of confocal scanning laser microscopy specific acquisition parameters on the detection and matching of speeded-up robust features. Ultramicroscopy 111(5), 364–374 (2011)
Stanciu, S.G., Hristu, R., Tranca, D.E., Stanciu, G.A.: Bags of features for classification of Laser Scanning Microscopy data. In: 2015 17th International Conference on Transparent Optical Networks (ICTON), pp. 1–4. IEEE (2015)
Stanciu, S.G., Xu, S., Peng, Q., Yan, J., Stanciu, G.A., Welsch, R.E., So, P.T., Csucs, G., Yu, H.: Experimenting liver fibrosis diagnostic by two photon excitation microscopy and bag-of-features image classification. Sci. Rep. 4, 4636 (2014)
Suhling, K., Hirvonen, L.M., Levitt, J.A., Chung, P.-H., Tregidgo, C., Le Marois, A., Rusakov, D.A., Zheng, K., Ameer-Beg, S., Poland, S.: Fluorescence lifetime imaging (FLIM): basic concepts and some recent developments. Med. Photonics 27, 3–40 (2015)
Tamaki, T., Yoshimuta, J., Kawakami, M., Raytchev, B., Kaneda, K., Yoshida, S., Takemura, Y., Onji, K., Miyaki, R., Tanaka, S.: Computer-aided colorectal tumor classification in NBI endoscopy using local features. Med. Image Anal. 17(1), 78–100 (2013)
Thomas, G., van Voskuilen, J., Truong, H., Song, J.-Y., Gerritsen, H.C., Sterenborg, H.: In vivo nonlinear spectral imaging as a tool to monitor early spectroscopic and metabolic changes in a murine cutaneous squamous cell carcinoma model. Biomed. Opt. Express 5(12), 4281–4299 (2014)
Tokarz, D., Cisek, R., Golaraei, A., Asa, S.L., Barzda, V., Wilson, B.C.: Ultrastructural features of collagen in thyroid carcinoma tissue observed by polarization second harmonic generation microscopy. Biomed. Opt. Express 6(9), 3475–3481 (2015)
Uckermann, O., Galli, R., Tamosaityte, S., Leipnitz, E., Geiger, K.D., Schackert, G., Koch, E., Steiner, G., Kirsch, M.: Label-free delineation of brain tumors by coherent anti-stokes Raman scattering microscopy in an orthotopic mouse model and human glioblastoma. PLoS One 9(9), e107115 (2014)
Xu, S., Fang, T., Li, D., Wang, S.: Object classification of aerial images with bag-of-visual words. Geosci. Remote Sens. Lett. IEEE 7(2), 366–370 (2010)
Zhou, L., Zhou, Z., Hu, D.: Scene classification using a multi-resolution bag-of-features model. Pattern Recogn. 46(1), 424–433 (2013)
Acknowledgments
The presented work was partially supported by the PN-II-RU-TE-2014-4-1803 and PN-II-PT-PCCA-2011-3.2-1162 Research Grants, funded by the Romanian Executive Agency for Higher Education, Research, Development and Innovation Funding (UEFISCDI). The corresponding author acknowledges as well the financial support of the SOP HRD, financed from the European Social Fund and the Romanian Government under the contract number POSDRU/159/1.5/S/137390/. The contribution of J.M. Bueno was supported by the Spanish SEIDI through the research grants FIS2013-41237-R and FIS2015-71933-REDT.
Author information
Authors and Affiliations
Corresponding author
Additional information
This article is part of the Topical Collection on Laser technologies and laser applications.
Guest Edited by José Figueiredo, José Rodrigues, Nikolai A. Sobolev, Paulo André and Rui Guerra.
Rights and permissions
About this article
Cite this article
Stanciu, S.G., Tranca, D.E., Stanciu, G.A. et al. Perspectives on combining Nonlinear Laser Scanning Microscopy and Bag-of-Features data classification strategies for automated disease diagnostics. Opt Quant Electron 48, 320 (2016). https://doi.org/10.1007/s11082-016-0589-8
Received:
Accepted:
Published:
DOI: https://doi.org/10.1007/s11082-016-0589-8