Skip to main content

Perspectives on combining Nonlinear Laser Scanning Microscopy and Bag-of-Features data classification strategies for automated disease diagnostics

Optical and Quantum Electronics volume 48, Article number: 320 (2016) Cite this article

Abstract

Nonlinear Laser Scanning Microscopy (NLSM) techniques have been demonstrated in the past two decades as powerful imaging tools for disease diagnostics (DD). Currently, in most DD related experiments the interpretation of NLSM data sets is performed by trained specialists. Such approaches are both time consuming and prone to errors due to inter- and intra-observer discrepancies. The Bag-of-Features (BoF) paradigm has demonstrated its potential usefulness with respect to automated data classification in the frame of multiple experiments, but its intersections with the field of NLSM are at this moment scarce, to say the least. In this paper we review recent progress on DD using NLSM, and discuss necessary steps and potential future perspectives for merging NLSM and BoF to achieve complex frameworks for automated DD with high sensitivity and specificity.

This is a preview of subscription content, access via your institution.

Access options

Buy single article

Instant access to the full article PDF.

USD 39.95

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Fig. 1

References

  • Alahi, A., Ortiz, R., Vandergheynst, P.: Freak: fast retina keypoint. In: 2012 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 510–517. IEEE (2012)

  • Aly, M., Munich, M., Perona, P.: Multiple dictionaries for bag of Words Large Scale Image Search. In: IEEE International Conference on Image Processing (ICIP) (2011)

  • Aucouturier, J.-J., Defreville, B., Pachet, F.: The bag-of-frames approach to audio pattern recognition: a sufficient model for urban soundscapes but not for polyphonic music. J. Acoust. Soc. Am. 122(2), 881–891 (2007)

    ADS  Article  Google Scholar 

  • Baenke, F., Peck, B., Miess, H., Schulze, A.: Hooked on fat: the role of lipid synthesis in cancer metabolism and tumour development. Dis. Models Mech. 6(6), 1353–1363 (2013)

    Article  Google Scholar 

  • Bay, H., Ess, A., Tuytelaars, T., Van Gool, L.: Speeded-up robust features (SURF). Comput. Vis. Image Underst. 110(3), 346–359 (2008)

    Article  Google Scholar 

  • Birk, J.W., Tadros, M., Moezardalan, K., Nadyarnykh, O., Forouhar, F., Anderson, J., Campagnola, P.: Second harmonic generation imaging distinguishes both high-grade dysplasia and cancer from normal colonic mucosa. Dig. Dis. Sci. 59(7), 1529–1534 (2014)

    Article  Google Scholar 

  • Bonnans, C., Chou, J., Werb, Z.: Remodelling the extracellular matrix in development and disease. Nat. Rev. Mol. Cell Biol. 15(12), 786–801 (2014)

    Article  Google Scholar 

  • Brelstaff, G., Bicego, M., Culeddu, N., Chessa, M.: Bag of peaks: interpretation of nmr spectrometry. Bioinformatics 25(2), 258–264 (2009)

    Article  Google Scholar 

  • Breunig, H.G., Weinigel, M., Bückle, R., Kellner-Höfer, M., Lademann, J., Darvin, M.E., Sterry, W., König, K.: Clinical coherent anti-Stokes Raman scattering and multiphoton tomography of human skin with a femtosecond laser and photonic crystal fiber. Laser Phys. Lett. 10(2), 025604 (2013)

    ADS  Article  Google Scholar 

  • Caicedo, J.C., Cruz, A., Gonzalez, F.A.: Histopathology image classification using bag of features and kernel functions. In: Artificial Intelligence in Medicine. pp. 126–135. Springer (2009)

  • Cao, Y., Wang, C., Li, Z., Zhang, L., Zhang, L.: Spatial-bag-of-features. In: 2010 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3352–3359. IEEE (2010)

  • Chakraborty, S., Nian, F.-S., Tsai, J.-W., Karmenyan, A., Chiou, A.: Quantification of the metabolic state in cell-model of Parkinson’s disease by fluorescence lifetime imaging microscopy. Sci. Rep. 6, 19145 (2016)

    ADS  Article  Google Scholar 

  • Chen, X., Nadiarynkh, O., Plotnikov, S., Campagnola, P.J.: Second harmonic generation microscopy for quantitative analysis of collagen fibrillar structure. Nat. Protoc. 7(4), 654–669 (2012)

    Article  Google Scholar 

  • Cicchi, R., Sturiale, A., Nesi, G., Kapsokalyvas, D., Alemanno, G., Tonelli, F., Pavone, F.S.: Multiphoton morpho-functional imaging of healthy colon mucosa, adenomatous polyp and adenocarcinoma. Biomed. Opt. Express 4(7), 1204–1213 (2013)

    Article  Google Scholar 

  • Dalal, N., Triggs, B.: Histograms of oriented gradients for human detection. In: Computer Vision and Pattern Recognition, 2005. CVPR 2005. In: Conference on IEEE Computer Society, pp. 886–893. IEEE (2005)

  • Evans, C.L., Xu, X., Kesari, S., Xie, X.S., Wong, S.T., Young, G.S.: Chemically-selective imaging of brain structures with CARS microscopy. Opt. Express 15(19), 12076–12087 (2007)

    ADS  Article  Google Scholar 

  • Eweiwi, A., Cheema, M.S., Bauckhage, C.: Action recognition in still images by learning spatial interest regions from videos. Pattern Recogn. Lett. 51, 8–15 (2015)

    Article  Google Scholar 

  • Fu, Z., Lu, G., Ting, K.M., Zhang, D.: Music classification via the bag-of-features approach. Pattern Recogn. Lett. 32(14), 1768–1777 (2011)

    Article  Google Scholar 

  • Gauderon, R., Lukins, P., Sheppard, C.: Second-harmonic generation imaging. In: Optics and Lasers in Biomedicine and Culture. pp. 66–69. Springer (2000)

  • Giacomelli, M.G., Sheikine, Y., Vardeh, H., Connolly, J.L., Fujimoto, J.G.: Rapid imaging of surgical breast excisions using direct temporal sampling two photon fluorescent lifetime imaging. Biomed. Opt. Express 6(11), 4317–4325 (2015)

    Article  Google Scholar 

  • Gulledge, C., Dewhirst, M.: Tumor oxygenation: a matter of supply and demand. Anticancer Res. 16(2), 741–749 (1995)

    Google Scholar 

  • Hamasha, K., Mohaidat, Q.I., Putnam, R.A., Woodman, R.C., Palchaudhuri, S., Rehse, S.J.: Sensitive and specific discrimination of pathogenic and nonpathogenic Escherichia coli using Raman spectroscopy—a comparison of two multivariate analysis techniques. Biomed. Opt. Express 4(4), 481–489 (2013)

    Article  Google Scholar 

  • Huang, X., Irmak, S., Lu, Y., Pipinos, I., Casale, G., Subbiah, J.: Spontaneous and coherent anti-Stokes Raman spectroscopy of human gastrocnemius muscle biopsies in CH-stretching region for discrimination of peripheral artery disease. Biomed. Opt. Express 6(8), 2766–2777 (2015)

    Article  Google Scholar 

  • Kantelhardt, S.R., Kalasauskas, D., König, K., Kim, E., Weinigel, M., Uchugonova, A., Giese, A.: In vivo multiphoton tomography and fluorescence lifetime imaging of human brain tumor tissue. J. Neuro-Oncol. 127(3), 473–482 (2016)

    Article  Google Scholar 

  • Kong, K., Kendall, C., Stone, N., Notingher, I.: Raman spectroscopy for medical diagnostics—From in vitro biofluid assays to in vivo cancer detection. Adv. Drug Deliv. Rev. 89, 121–134 (2015)

    Article  Google Scholar 

  • Krahmer, N., Farese, R.V., Walther, T.C.: Balancing the fat: lipid droplets and human disease. EMBO Mol. Med. 5(7), 973–983 (2013)

    Article  Google Scholar 

  • Lazebnik, S., Schmid, C., Ponce, J.: Beyond bags of features: Spatial pyramid matching for recognizing natural scene categories. In: 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 2169–2178. IEEE (2006)

  • Leutenegger, S., Chli, M., Siegwart, R.Y.: BRISK: Binary robust invariant scalable keypoints. In: 2011 IEEE International Conference on Computer Vision (ICCV), pp. 2548–2555. IEEE (2011)

  • Li, L., Chen, Z., Wang, X., Liu, X., Jiang, W., Zhuo, S., Guan, G., Chen, J.: Visualization of tumor response to neoadjuvant therapy for rectal carcinoma by nonlinear optical imaging. IEEE J. Sel. Top. Quantum Electron. 22(3), 6800206 (2016)

    Article  Google Scholar 

  • Li L, Li H, Chen Z, Zhuo S, Feng C, Yang Y, Guan G, Chen J Layer-resolved colorectal tissues using nonlinear microscopy. Lasers Med. Sci. 1–9 (2015)

  • Lowe, D.G.: Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vision 60(2), 91–110 (2004)

    Article  Google Scholar 

  • Lu, P., Weaver, V.M., Werb, Z.: The extracellular matrix: a dynamic niche in cancer progression. J. Cell Biol. 196(4), 395–406 (2012)

    Article  Google Scholar 

  • Makino, T., Jain, M., Montrose, D.C., Aggarwal, A., Sterling, J., Bosworth, B.P., Milsom, J.W., Robinson, B.D., Shevchuk, M.M., Kawaguchi, K.: Multiphoton tomographic imaging: a potential optical biopsy tool for detecting gastrointestinal inflammation and neoplasia. Cancer Prev. Res. 5(11), 1280–1290 (2012)

    Article  Google Scholar 

  • Nanni, L., Costa, Y.M., Lumini, A., Kim, M.Y., Baek, S.R.: Combining visual and acoustic features for music genre classification. Expert Syst. Appl. 45, 108–117 (2016)

    Article  Google Scholar 

  • Nanni, L., Melucci, M.: Combination of projectors, standard texture descriptors and bag of features for classifying images. Neurocomputing 173, 1602–1614 (2016)

    Article  Google Scholar 

  • Nordestgaard, B.G., Varbo, A.: Triglycerides and cardiovascular disease. Lancet 384(9943), 626–635 (2014)

    Article  Google Scholar 

  • O’Hara, S., Draper, B.A.: Introduction to the bag of features paradigm for image classification and retrieval. arXiv preprint arXiv:1101.3354 (2011)

  • Pezacki, J.P., Blake, J.A., Danielson, D.C., Kennedy, D.C., Lyn, R.K., Singaravelu, R.: Chemical contrast for imaging living systems: molecular vibrations drive CARS microscopy. Nat. Chem. Biol. 7(3), 137–145 (2011)

    Article  Google Scholar 

  • Raykar, V.C., Yu, S., Zhao, L.H., Jerebko, A., Florin, C., Valadez, G.H., Bogoni, L., Moy, L.: Supervised learning from multiple experts: whom to trust when everyone lies a bit. In: Proceedings of the 26th Annual international conference on machine learning, pp. 889–896. ACM (2009)

  • Rublee, E., Rabaud, V., Konolige, K., Bradski, G.: ORB: an efficient alternative to SIFT or SURF. In: 2011 IEEE International Conference on Computer Vision (ICCV), pp. 2564–2571. IEEE (2011)

  • Sadek, I., Sidibé, D., Meriaudeau, F.: Automatic discrimination of color retinal images using the bag of words approach. In: SPIE Medical Imaging 2015, pp. 94141 J-94141 J-94148. International Society for Optics and Photonics

  • Schie, I.W., Krafft, C., Popp, J.: Applications of coherent Raman scattering microscopies to clinical and biological studies. Analyst 140(12), 3897–3909 (2015)

    Article  Google Scholar 

  • So, P.T., Dong, C.Y., Masters, B.R., Berland, K.M.: Two-photon excitation fluorescence microscopy. Annu. Rev. Biomed. Eng. 2(1), 399–429 (2000)

    Article  Google Scholar 

  • Stanciu, S.G., Coltuc, D., Tranca, D.E., Stanciu, G.A.: Matching DSIFT descriptors extracted from CSLM images. Engineering 5(10), 199–202 (2013)

    Article  Google Scholar 

  • Stanciu, S.G., Hristu, R., Boriga, R., Stanciu, G.A.: On the suitability of SIFT technique to deal with image modifications specific to Confocal Scanning Laser Microscopy. Microsc. Microanal. 16(05), 515–530 (2010)

    ADS  Article  Google Scholar 

  • Stanciu, S.G., Hristu, R., Stanciu, G.A.: Influence of confocal scanning laser microscopy specific acquisition parameters on the detection and matching of speeded-up robust features. Ultramicroscopy 111(5), 364–374 (2011)

    Article  Google Scholar 

  • Stanciu, S.G., Hristu, R., Tranca, D.E., Stanciu, G.A.: Bags of features for classification of Laser Scanning Microscopy data. In: 2015 17th International Conference on Transparent Optical Networks (ICTON), pp. 1–4. IEEE (2015)

  • Stanciu, S.G., Xu, S., Peng, Q., Yan, J., Stanciu, G.A., Welsch, R.E., So, P.T., Csucs, G., Yu, H.: Experimenting liver fibrosis diagnostic by two photon excitation microscopy and bag-of-features image classification. Sci. Rep. 4, 4636 (2014)

    ADS  Article  Google Scholar 

  • Suhling, K., Hirvonen, L.M., Levitt, J.A., Chung, P.-H., Tregidgo, C., Le Marois, A., Rusakov, D.A., Zheng, K., Ameer-Beg, S., Poland, S.: Fluorescence lifetime imaging (FLIM): basic concepts and some recent developments. Med. Photonics 27, 3–40 (2015)

    Article  Google Scholar 

  • Tamaki, T., Yoshimuta, J., Kawakami, M., Raytchev, B., Kaneda, K., Yoshida, S., Takemura, Y., Onji, K., Miyaki, R., Tanaka, S.: Computer-aided colorectal tumor classification in NBI endoscopy using local features. Med. Image Anal. 17(1), 78–100 (2013)

    Article  Google Scholar 

  • Thomas, G., van Voskuilen, J., Truong, H., Song, J.-Y., Gerritsen, H.C., Sterenborg, H.: In vivo nonlinear spectral imaging as a tool to monitor early spectroscopic and metabolic changes in a murine cutaneous squamous cell carcinoma model. Biomed. Opt. Express 5(12), 4281–4299 (2014)

    Article  Google Scholar 

  • Tokarz, D., Cisek, R., Golaraei, A., Asa, S.L., Barzda, V., Wilson, B.C.: Ultrastructural features of collagen in thyroid carcinoma tissue observed by polarization second harmonic generation microscopy. Biomed. Opt. Express 6(9), 3475–3481 (2015)

    Article  Google Scholar 

  • Uckermann, O., Galli, R., Tamosaityte, S., Leipnitz, E., Geiger, K.D., Schackert, G., Koch, E., Steiner, G., Kirsch, M.: Label-free delineation of brain tumors by coherent anti-stokes Raman scattering microscopy in an orthotopic mouse model and human glioblastoma. PLoS One 9(9), e107115 (2014)

    ADS  Article  Google Scholar 

  • Xu, S., Fang, T., Li, D., Wang, S.: Object classification of aerial images with bag-of-visual words. Geosci. Remote Sens. Lett. IEEE 7(2), 366–370 (2010)

    ADS  Article  Google Scholar 

  • Zhou, L., Zhou, Z., Hu, D.: Scene classification using a multi-resolution bag-of-features model. Pattern Recogn. 46(1), 424–433 (2013)

    Article  Google Scholar 

Download references

Acknowledgments

The presented work was partially supported by the PN-II-RU-TE-2014-4-1803 and PN-II-PT-PCCA-2011-3.2-1162 Research Grants, funded by the Romanian Executive Agency for Higher Education, Research, Development and Innovation Funding (UEFISCDI). The corresponding author acknowledges as well the financial support of the SOP HRD, financed from the European Social Fund and the Romanian Government under the contract number POSDRU/159/1.5/S/137390/. The contribution of J.M. Bueno was supported by the Spanish SEIDI through the research grants FIS2013-41237-R and FIS2015-71933-REDT.

Author information

Authors and Affiliations

  1. Center for Microscopy-Microanalysis and Information Processing, University Politehnica of Bucharest, 313 Splaiul Independentei Bvd., 060042, Bucharest, Romania

    Stefan G. Stanciu, Denis E. Tranca, George A. Stanciu & Radu Hristu

  2. Laboratorio de Óptica, Universidad de Murcia, Campus Espinardo, 30100, Murcia, Spain

    Juan M. Bueno

Authors
  1. Stefan G. Stanciu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Denis E. Tranca
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. George A. Stanciu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Radu Hristu
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Juan M. Bueno
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Stefan G. Stanciu.

Additional information

This article is part of the Topical Collection on Laser technologies and laser applications.

Guest Edited by José Figueiredo, José Rodrigues, Nikolai A. Sobolev, Paulo André and Rui Guerra.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Stanciu, S.G., Tranca, D.E., Stanciu, G.A. et al. Perspectives on combining Nonlinear Laser Scanning Microscopy and Bag-of-Features data classification strategies for automated disease diagnostics. Opt Quant Electron 48, 320 (2016). https://doi.org/10.1007/s11082-016-0589-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s11082-016-0589-8

Keywords

  • Nonlinear Laser Scanning Microscopy
  • Two-photon excitation fluorescence microscopy
  • Second harmonic generation microscopy
  • Fluorescence lifetime imaging
  • Coherent anti-stokes Raman microscopy
  • Bag-of-Features
  • Data classification
  • Data retrieval