Skip to main content

Perspectives on combining Nonlinear Laser Scanning Microscopy and Bag-of-Features data classification strategies for automated disease diagnostics

Abstract

Nonlinear Laser Scanning Microscopy (NLSM) techniques have been demonstrated in the past two decades as powerful imaging tools for disease diagnostics (DD). Currently, in most DD related experiments the interpretation of NLSM data sets is performed by trained specialists. Such approaches are both time consuming and prone to errors due to inter- and intra-observer discrepancies. The Bag-of-Features (BoF) paradigm has demonstrated its potential usefulness with respect to automated data classification in the frame of multiple experiments, but its intersections with the field of NLSM are at this moment scarce, to say the least. In this paper we review recent progress on DD using NLSM, and discuss necessary steps and potential future perspectives for merging NLSM and BoF to achieve complex frameworks for automated DD with high sensitivity and specificity.

This is a preview of subscription content, access via your institution.

Fig. 1

References

  1. Alahi, A., Ortiz, R., Vandergheynst, P.: Freak: fast retina keypoint. In: 2012 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 510–517. IEEE (2012)

  2. Aly, M., Munich, M., Perona, P.: Multiple dictionaries for bag of Words Large Scale Image Search. In: IEEE International Conference on Image Processing (ICIP) (2011)

  3. Aucouturier, J.-J., Defreville, B., Pachet, F.: The bag-of-frames approach to audio pattern recognition: a sufficient model for urban soundscapes but not for polyphonic music. J. Acoust. Soc. Am. 122(2), 881–891 (2007)

    ADS  Article  Google Scholar 

  4. Baenke, F., Peck, B., Miess, H., Schulze, A.: Hooked on fat: the role of lipid synthesis in cancer metabolism and tumour development. Dis. Models Mech. 6(6), 1353–1363 (2013)

    Article  Google Scholar 

  5. Bay, H., Ess, A., Tuytelaars, T., Van Gool, L.: Speeded-up robust features (SURF). Comput. Vis. Image Underst. 110(3), 346–359 (2008)

    Article  Google Scholar 

  6. Birk, J.W., Tadros, M., Moezardalan, K., Nadyarnykh, O., Forouhar, F., Anderson, J., Campagnola, P.: Second harmonic generation imaging distinguishes both high-grade dysplasia and cancer from normal colonic mucosa. Dig. Dis. Sci. 59(7), 1529–1534 (2014)

    Article  Google Scholar 

  7. Bonnans, C., Chou, J., Werb, Z.: Remodelling the extracellular matrix in development and disease. Nat. Rev. Mol. Cell Biol. 15(12), 786–801 (2014)

    Article  Google Scholar 

  8. Brelstaff, G., Bicego, M., Culeddu, N., Chessa, M.: Bag of peaks: interpretation of nmr spectrometry. Bioinformatics 25(2), 258–264 (2009)

    Article  Google Scholar 

  9. Breunig, H.G., Weinigel, M., Bückle, R., Kellner-Höfer, M., Lademann, J., Darvin, M.E., Sterry, W., König, K.: Clinical coherent anti-Stokes Raman scattering and multiphoton tomography of human skin with a femtosecond laser and photonic crystal fiber. Laser Phys. Lett. 10(2), 025604 (2013)

    ADS  Article  Google Scholar 

  10. Caicedo, J.C., Cruz, A., Gonzalez, F.A.: Histopathology image classification using bag of features and kernel functions. In: Artificial Intelligence in Medicine. pp. 126–135. Springer (2009)

  11. Cao, Y., Wang, C., Li, Z., Zhang, L., Zhang, L.: Spatial-bag-of-features. In: 2010 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3352–3359. IEEE (2010)

  12. Chakraborty, S., Nian, F.-S., Tsai, J.-W., Karmenyan, A., Chiou, A.: Quantification of the metabolic state in cell-model of Parkinson’s disease by fluorescence lifetime imaging microscopy. Sci. Rep. 6, 19145 (2016)

    ADS  Article  Google Scholar 

  13. Chen, X., Nadiarynkh, O., Plotnikov, S., Campagnola, P.J.: Second harmonic generation microscopy for quantitative analysis of collagen fibrillar structure. Nat. Protoc. 7(4), 654–669 (2012)

    Article  Google Scholar 

  14. Cicchi, R., Sturiale, A., Nesi, G., Kapsokalyvas, D., Alemanno, G., Tonelli, F., Pavone, F.S.: Multiphoton morpho-functional imaging of healthy colon mucosa, adenomatous polyp and adenocarcinoma. Biomed. Opt. Express 4(7), 1204–1213 (2013)

    Article  Google Scholar 

  15. Dalal, N., Triggs, B.: Histograms of oriented gradients for human detection. In: Computer Vision and Pattern Recognition, 2005. CVPR 2005. In: Conference on IEEE Computer Society, pp. 886–893. IEEE (2005)

  16. Evans, C.L., Xu, X., Kesari, S., Xie, X.S., Wong, S.T., Young, G.S.: Chemically-selective imaging of brain structures with CARS microscopy. Opt. Express 15(19), 12076–12087 (2007)

    ADS  Article  Google Scholar 

  17. Eweiwi, A., Cheema, M.S., Bauckhage, C.: Action recognition in still images by learning spatial interest regions from videos. Pattern Recogn. Lett. 51, 8–15 (2015)

    Article  Google Scholar 

  18. Fu, Z., Lu, G., Ting, K.M., Zhang, D.: Music classification via the bag-of-features approach. Pattern Recogn. Lett. 32(14), 1768–1777 (2011)

    Article  Google Scholar 

  19. Gauderon, R., Lukins, P., Sheppard, C.: Second-harmonic generation imaging. In: Optics and Lasers in Biomedicine and Culture. pp. 66–69. Springer (2000)

  20. Giacomelli, M.G., Sheikine, Y., Vardeh, H., Connolly, J.L., Fujimoto, J.G.: Rapid imaging of surgical breast excisions using direct temporal sampling two photon fluorescent lifetime imaging. Biomed. Opt. Express 6(11), 4317–4325 (2015)

    Article  Google Scholar 

  21. Gulledge, C., Dewhirst, M.: Tumor oxygenation: a matter of supply and demand. Anticancer Res. 16(2), 741–749 (1995)

    Google Scholar 

  22. Hamasha, K., Mohaidat, Q.I., Putnam, R.A., Woodman, R.C., Palchaudhuri, S., Rehse, S.J.: Sensitive and specific discrimination of pathogenic and nonpathogenic Escherichia coli using Raman spectroscopy—a comparison of two multivariate analysis techniques. Biomed. Opt. Express 4(4), 481–489 (2013)

    Article  Google Scholar 

  23. Huang, X., Irmak, S., Lu, Y., Pipinos, I., Casale, G., Subbiah, J.: Spontaneous and coherent anti-Stokes Raman spectroscopy of human gastrocnemius muscle biopsies in CH-stretching region for discrimination of peripheral artery disease. Biomed. Opt. Express 6(8), 2766–2777 (2015)

    Article  Google Scholar 

  24. Kantelhardt, S.R., Kalasauskas, D., König, K., Kim, E., Weinigel, M., Uchugonova, A., Giese, A.: In vivo multiphoton tomography and fluorescence lifetime imaging of human brain tumor tissue. J. Neuro-Oncol. 127(3), 473–482 (2016)

    Article  Google Scholar 

  25. Kong, K., Kendall, C., Stone, N., Notingher, I.: Raman spectroscopy for medical diagnostics—From in vitro biofluid assays to in vivo cancer detection. Adv. Drug Deliv. Rev. 89, 121–134 (2015)

    Article  Google Scholar 

  26. Krahmer, N., Farese, R.V., Walther, T.C.: Balancing the fat: lipid droplets and human disease. EMBO Mol. Med. 5(7), 973–983 (2013)

    Article  Google Scholar 

  27. Lazebnik, S., Schmid, C., Ponce, J.: Beyond bags of features: Spatial pyramid matching for recognizing natural scene categories. In: 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 2169–2178. IEEE (2006)

  28. Leutenegger, S., Chli, M., Siegwart, R.Y.: BRISK: Binary robust invariant scalable keypoints. In: 2011 IEEE International Conference on Computer Vision (ICCV), pp. 2548–2555. IEEE (2011)

  29. Li, L., Chen, Z., Wang, X., Liu, X., Jiang, W., Zhuo, S., Guan, G., Chen, J.: Visualization of tumor response to neoadjuvant therapy for rectal carcinoma by nonlinear optical imaging. IEEE J. Sel. Top. Quantum Electron. 22(3), 6800206 (2016)

    Article  Google Scholar 

  30. Li L, Li H, Chen Z, Zhuo S, Feng C, Yang Y, Guan G, Chen J Layer-resolved colorectal tissues using nonlinear microscopy. Lasers Med. Sci. 1–9 (2015)

  31. Lowe, D.G.: Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vision 60(2), 91–110 (2004)

    Article  Google Scholar 

  32. Lu, P., Weaver, V.M., Werb, Z.: The extracellular matrix: a dynamic niche in cancer progression. J. Cell Biol. 196(4), 395–406 (2012)

    Article  Google Scholar 

  33. Makino, T., Jain, M., Montrose, D.C., Aggarwal, A., Sterling, J., Bosworth, B.P., Milsom, J.W., Robinson, B.D., Shevchuk, M.M., Kawaguchi, K.: Multiphoton tomographic imaging: a potential optical biopsy tool for detecting gastrointestinal inflammation and neoplasia. Cancer Prev. Res. 5(11), 1280–1290 (2012)

    Article  Google Scholar 

  34. Nanni, L., Costa, Y.M., Lumini, A., Kim, M.Y., Baek, S.R.: Combining visual and acoustic features for music genre classification. Expert Syst. Appl. 45, 108–117 (2016)

    Article  Google Scholar 

  35. Nanni, L., Melucci, M.: Combination of projectors, standard texture descriptors and bag of features for classifying images. Neurocomputing 173, 1602–1614 (2016)

    Article  Google Scholar 

  36. Nordestgaard, B.G., Varbo, A.: Triglycerides and cardiovascular disease. Lancet 384(9943), 626–635 (2014)

    Article  Google Scholar 

  37. O’Hara, S., Draper, B.A.: Introduction to the bag of features paradigm for image classification and retrieval. arXiv preprint arXiv:1101.3354 (2011)

  38. Pezacki, J.P., Blake, J.A., Danielson, D.C., Kennedy, D.C., Lyn, R.K., Singaravelu, R.: Chemical contrast for imaging living systems: molecular vibrations drive CARS microscopy. Nat. Chem. Biol. 7(3), 137–145 (2011)

    Article  Google Scholar 

  39. Raykar, V.C., Yu, S., Zhao, L.H., Jerebko, A., Florin, C., Valadez, G.H., Bogoni, L., Moy, L.: Supervised learning from multiple experts: whom to trust when everyone lies a bit. In: Proceedings of the 26th Annual international conference on machine learning, pp. 889–896. ACM (2009)

  40. Rublee, E., Rabaud, V., Konolige, K., Bradski, G.: ORB: an efficient alternative to SIFT or SURF. In: 2011 IEEE International Conference on Computer Vision (ICCV), pp. 2564–2571. IEEE (2011)

  41. Sadek, I., Sidibé, D., Meriaudeau, F.: Automatic discrimination of color retinal images using the bag of words approach. In: SPIE Medical Imaging 2015, pp. 94141 J-94141 J-94148. International Society for Optics and Photonics

  42. Schie, I.W., Krafft, C., Popp, J.: Applications of coherent Raman scattering microscopies to clinical and biological studies. Analyst 140(12), 3897–3909 (2015)

    Article  Google Scholar 

  43. So, P.T., Dong, C.Y., Masters, B.R., Berland, K.M.: Two-photon excitation fluorescence microscopy. Annu. Rev. Biomed. Eng. 2(1), 399–429 (2000)

    Article  Google Scholar 

  44. Stanciu, S.G., Coltuc, D., Tranca, D.E., Stanciu, G.A.: Matching DSIFT descriptors extracted from CSLM images. Engineering 5(10), 199–202 (2013)

    Article  Google Scholar 

  45. Stanciu, S.G., Hristu, R., Boriga, R., Stanciu, G.A.: On the suitability of SIFT technique to deal with image modifications specific to Confocal Scanning Laser Microscopy. Microsc. Microanal. 16(05), 515–530 (2010)

    ADS  Article  Google Scholar 

  46. Stanciu, S.G., Hristu, R., Stanciu, G.A.: Influence of confocal scanning laser microscopy specific acquisition parameters on the detection and matching of speeded-up robust features. Ultramicroscopy 111(5), 364–374 (2011)

    Article  Google Scholar 

  47. Stanciu, S.G., Hristu, R., Tranca, D.E., Stanciu, G.A.: Bags of features for classification of Laser Scanning Microscopy data. In: 2015 17th International Conference on Transparent Optical Networks (ICTON), pp. 1–4. IEEE (2015)

  48. Stanciu, S.G., Xu, S., Peng, Q., Yan, J., Stanciu, G.A., Welsch, R.E., So, P.T., Csucs, G., Yu, H.: Experimenting liver fibrosis diagnostic by two photon excitation microscopy and bag-of-features image classification. Sci. Rep. 4, 4636 (2014)

    ADS  Article  Google Scholar 

  49. Suhling, K., Hirvonen, L.M., Levitt, J.A., Chung, P.-H., Tregidgo, C., Le Marois, A., Rusakov, D.A., Zheng, K., Ameer-Beg, S., Poland, S.: Fluorescence lifetime imaging (FLIM): basic concepts and some recent developments. Med. Photonics 27, 3–40 (2015)

    Article  Google Scholar 

  50. Tamaki, T., Yoshimuta, J., Kawakami, M., Raytchev, B., Kaneda, K., Yoshida, S., Takemura, Y., Onji, K., Miyaki, R., Tanaka, S.: Computer-aided colorectal tumor classification in NBI endoscopy using local features. Med. Image Anal. 17(1), 78–100 (2013)

    Article  Google Scholar 

  51. Thomas, G., van Voskuilen, J., Truong, H., Song, J.-Y., Gerritsen, H.C., Sterenborg, H.: In vivo nonlinear spectral imaging as a tool to monitor early spectroscopic and metabolic changes in a murine cutaneous squamous cell carcinoma model. Biomed. Opt. Express 5(12), 4281–4299 (2014)

    Article  Google Scholar 

  52. Tokarz, D., Cisek, R., Golaraei, A., Asa, S.L., Barzda, V., Wilson, B.C.: Ultrastructural features of collagen in thyroid carcinoma tissue observed by polarization second harmonic generation microscopy. Biomed. Opt. Express 6(9), 3475–3481 (2015)

    Article  Google Scholar 

  53. Uckermann, O., Galli, R., Tamosaityte, S., Leipnitz, E., Geiger, K.D., Schackert, G., Koch, E., Steiner, G., Kirsch, M.: Label-free delineation of brain tumors by coherent anti-stokes Raman scattering microscopy in an orthotopic mouse model and human glioblastoma. PLoS One 9(9), e107115 (2014)

    ADS  Article  Google Scholar 

  54. Xu, S., Fang, T., Li, D., Wang, S.: Object classification of aerial images with bag-of-visual words. Geosci. Remote Sens. Lett. IEEE 7(2), 366–370 (2010)

    ADS  Article  Google Scholar 

  55. Zhou, L., Zhou, Z., Hu, D.: Scene classification using a multi-resolution bag-of-features model. Pattern Recogn. 46(1), 424–433 (2013)

    Article  Google Scholar 

Download references

Acknowledgments

The presented work was partially supported by the PN-II-RU-TE-2014-4-1803 and PN-II-PT-PCCA-2011-3.2-1162 Research Grants, funded by the Romanian Executive Agency for Higher Education, Research, Development and Innovation Funding (UEFISCDI). The corresponding author acknowledges as well the financial support of the SOP HRD, financed from the European Social Fund and the Romanian Government under the contract number POSDRU/159/1.5/S/137390/. The contribution of J.M. Bueno was supported by the Spanish SEIDI through the research grants FIS2013-41237-R and FIS2015-71933-REDT.

Author information

Affiliations

Authors

Corresponding author

Correspondence to Stefan G. Stanciu.

Additional information

This article is part of the Topical Collection on Laser technologies and laser applications.

Guest Edited by José Figueiredo, José Rodrigues, Nikolai A. Sobolev, Paulo André and Rui Guerra.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Stanciu, S.G., Tranca, D.E., Stanciu, G.A. et al. Perspectives on combining Nonlinear Laser Scanning Microscopy and Bag-of-Features data classification strategies for automated disease diagnostics. Opt Quant Electron 48, 320 (2016). https://doi.org/10.1007/s11082-016-0589-8

Download citation

Keywords

  • Nonlinear Laser Scanning Microscopy
  • Two-photon excitation fluorescence microscopy
  • Second harmonic generation microscopy
  • Fluorescence lifetime imaging
  • Coherent anti-stokes Raman microscopy
  • Bag-of-Features
  • Data classification
  • Data retrieval