Abstract
Spatially resolved transcriptomics (SRT) is a vital technique in biology that allows for gene expression measurement at the resolution of individual spots while preserving spatial information. However, owing to technical limitations, single-spot resolution often includes data from multiple cells, leading to suboptimal results and opportunities for improvement. In this study, we propose a deep learning-based, plug-and-play method for enhancing spot resolution to obtain higher-resolution SRT data. Our approach involves training a convolutional neural network (CNN) model and introducing a shift-predict operation to obtain superresolution spots. Using a human breast cancer SRT dataset, we demonstrate that our method achieves 9 × superresolution, outperforming traditional superresolution techniques. Crucially, our method decreased the mean squared error (MSE) to 1.379 for all genes, 2.287 for tumor-related genes at 4 × superresolution, 1.866 for all genes, and 3.371 for tumor-related genes at 9 × superresolution, reflecting substantial improvements compared to the traditional approaches, including Gaussian RBF, multiquadric RBF, linear RBF, resize-predict, bilinear, and bicubic methods. Furthermore, we verify our method's effectiveness using external and simulated datasets. Our proposed method offers a substantial advancement in SRT by enabling higher-resolution gene expression data generation. By providing a deeper understanding of gene expression patterns and their underlying biological significance, this method contributes to progress in biology and medicine.
Similar content being viewed by others
Data availability
The human breast cancer in situ capturing transcriptomics we used in this paper is available at https://data.mendeley.com/datasets/29ntw7sh4r. The Breast Cancer Semantic Segmentation (BCSS) dataset utilized is accessible at https://github.com/PathologyDataScience/BCSS.
Abbreviations
- BCSS:
-
Breast cancer semantic segmentation
- CNN:
-
Convolutional neural network
- GAP:
-
Global average pooling
- HVS:
-
Human visual system
- MSE:
-
Mean square error
- NMF:
-
Nonnegative matrix factorization
- PSNR:
-
Peak-signal-to-noise ratio
- RBF:
-
Radial basis function
- ROI:
-
Region of interest
- scRNA-seq:
-
Single-cell RNA sequencing
- SP:
-
Senior pathologist
- SRT:
-
Spatially resolved transcriptomics
- SSIM:
-
Structural similarity index measure
- WSIs:
-
Whole-slide images
References
Rao A, Barkley D, França GS, Yanai I (2021) Exploring tissue architecture using spatial transcriptomics. Nature 596:211–220. https://doi.org/10.1038/s41586-021-03634-9
Tang F, Barbacioru C, Wang Y, Nordman E, Lee C, Xu N, Wang X, Bodeau J, Tuch BB, Siddiqui A, Lao K, Surani MA (2009) mRNA-Seq whole-transcriptome analysis of a single cell. Nat Methods 6:377–382. https://doi.org/10.1038/nmeth.1315
Macosko EZ, Basu A, Satija R, Nemesh J, Shekhar K, Goldman M, Tirosh I, Bialas AR, Kamitaki N, Martersteck EM, Trombetta JJ, Weitz DA, Sanes JR, Shalek AK, Regev A, McCarroll SA (2015) Highly parallel genome-wide expression profiling of individual cells using nanoliter droplets. Cell 161:1202–1214. https://doi.org/10.1016/j.cell.2015.05.002
Zheng GXY, Terry JM, Belgrader P, Ryvkin P, Bent ZW, Wilson R, Ziraldo SB, Wheeler TD, McDermott GP, Zhu J, Gregory MT, Shuga J, Montesclaros L, Underwood JG, Masquelier DA, Nishimura SY, Schnall-Levin M, Wyatt PW, Hindson CM, Bharadwaj R, Wong A, Ness KD, Beppu LW, Deeg HJ, McFarland C, Loeb KR, Valente WJ, Ericson NG, Stevens EA, Radich JP, Mikkelsen TS, Hindson BJ, Bielas JH (2017) Massively parallel digital transcriptional profiling of single cells. Nat Commun 8:14049. https://doi.org/10.1038/ncomms14049
Marx V (2021) Method of the year: spatially resolved transcriptomics. Nat Methods 18:9–14. https://doi.org/10.1038/s41592-020-01033-y
Eng C-HL, Lawson M, Zhu Q, Dries R, Koulena N, Takei Y, Yun J, Cronin C, Karp C, Yuan G-C, Cai L (2019) Transcriptome-scale super-resolved imaging in tissues by RNA seqFISH+. Nature 568:235–239. https://doi.org/10.1038/s41586-019-1049-y
Codeluppi S, Borm LE, Zeisel A, La Manno G, van Lunteren JA, Svensson CI, Linnarsson S (2018) Spatial organization of the somatosensory cortex revealed by osmFISH. Nat Methods 15:932–935. https://doi.org/10.1038/s41592-018-0175-z
Moses L, Pachter L (2022) Museum of spatial transcriptomics. Nat Methods 19:534–546. https://doi.org/10.1038/s41592-022-01409-2
Rodriques SG, Stickels RR, Goeva A, Martin CA, Murray E, Vanderburg CR, Welch J, Chen LM, Chen F, Macosko EZ (2019) Slide-seq: a scalable technology for measuring genome-wide expression at high spatial resolution. Science 363:1463–1467. https://doi.org/10.1126/science.aaw1219
Stickels RR, Murray E, Kumar P, Li J, Marshall JL, Di Bella DJ, Arlotta P, Macosko EZ, Chen F (2021) Highly sensitive spatial transcriptomics at near-cellular resolution with Slide-seqV2. Nat Biotechnol 39:313–319. https://doi.org/10.1038/s41587-020-0739-1
Yue L, Liu F, Hu J, Yang P, Wang Y, Dong J, Shu W, Huang X, Wang S (2023) A guidebook of spatial transcriptomic technologies, data resources and analysis approaches. Comput Struct Biotechnol J 21:940–955. https://doi.org/10.1016/j.csbj.2023.01.016
Ståhl PL, Salmén F, Vickovic S, Lundmark A, Navarro JF, Magnusson J, Giacomello S, Asp M, Westholm JO, Huss M, Mollbrink A, Linnarsson S, Codeluppi S, Borg Å, Pontén F, Costea PI, Sahlén P, Mulder J, Bergmann O, Lundeberg J, Frisén J (2016) Visualization and analysis of gene expression in tissue sections by spatial transcriptomics. Science 353:78–82. https://doi.org/10.1126/science.aaf2403
Maynard KR, Collado-Torres L, Weber LM, Uytingco C, Barry BK, Williams SR, Catallini JL, Tran MN, Besich Z, Tippani M, Chew J, Yin Y, Kleinman JE, Hyde TM, Rao N, Hicks SC, Martinowich K, Jaffe AE (2021) Transcriptome-scale spatial gene expression in the human dorsolateral prefrontal cortex. Nat Neurosci 24:425–436. https://doi.org/10.1038/s41593-020-00787-0
Yang F, Wang W, Wang F, Fang Y, Tang D, Huang J, Lu H, Yao J (2022) scBERT as a large-scale pretrained deep language model for cell type annotation of single-cell RNA-seq data. Nat Mach Intell 4:852–866. https://doi.org/10.1038/s42256-022-00534-z
Newman AM, Steen CB, Liu CL, Gentles AJ, Chaudhuri AA, Scherer F, Khodadoust MS, Esfahani MS, Luca BA, Steiner D, Diehn M, Alizadeh AA (2019) Determining cell type abundance and expression from bulk tissues with digital cytometry. Nat Biotechnol 37:773–782. https://doi.org/10.1038/s41587-019-0114-2
Li L, Zhang Y, Ren Y et al (2022) Pan-cancer single-cell analysis reveals the core factors and pathway in specific cancer stem cells of upper gastrointestinal cancer. Front Bioeng Biotechnol 10:1–12. https://doi.org/10.3389/fbioe.2022.849798
Huang Y, Zhang P (2021) Evaluation of machine learning approaches for cell-type identification from single-cell transcriptomics data. Brief Bioinform 22:bbab035. https://doi.org/10.1093/bib/bbab035
Kong Y, Genchev GZ, Wang X, Zhao H, Lu H (2020) Nuclear segmentation in histopathological images using two-stage stacked U-Nets with attention mechanism. Front Bioeng Biotechnol 8:573866. https://doi.org/10.3389/fbioe.2020.573866
Coudray N, Ocampo PS, Sakellaropoulos T, Narula N, Snuderl M, Fenyö D, Moreira AL, Razavian N, Tsirigos A (2018) Classification and mutation prediction from non–small cell lung cancer histopathology images using deep learning. Nat Med 24:1559–1567. https://doi.org/10.1038/s41591-018-0177-5
Gürünlü B, Öztürk S (2022) A novel method for forgery detection on lung cancer images. IJISS 11:13–20
Xiao X, Kong Y, Wang Z, Lu H (2023) Transformer with convolution and graph-node co-embedding: an accurate and interpretable vision backbone for predicting gene expressions from local histopathological image. bioRxiv. https://doi.org/10.1101/2023.05.28.542669
Cable DM, Murray E, Zou LS, Goeva A, Macosko EZ, Chen F, Irizarry RA (2022) Robust decomposition of cell type mixtures in spatial transcriptomics. Nat Biotechnol 40:517–526. https://doi.org/10.1038/s41587-021-00830-w
Dong R, Yuan G-C (2021) SpatialDWLS: accurate deconvolution of spatial transcriptomic data. Genome Biol 22:145. https://doi.org/10.1186/s13059-021-02362-7
Elosua-Bayes M, Nieto P, Mereu E, Gut I, Heyn H (2021) SPOTlight: seeded NMF regression to deconvolute spatial transcriptomics spots with single-cell transcriptomes. Nucleic Acids Res 49:e50. https://doi.org/10.1093/nar/gkab043
Biancalani T, Scalia G, Buffoni L, Avasthi R, Lu Z, Sanger A, Tokcan N, Vanderburg CR, Segerstolpe Å, Zhang M, Avraham-Davidi I, Vickovic S, Nitzan M, Ma S, Subramanian A, Lipinski M, Buenrostro J, Brown NB, Fanelli D, Zhuang X, Macosko EZ, Regev A (2021) Deep learning and alignment of spatially resolved single-cell transcriptomes with Tangram. Nat Methods 18:1352–1362. https://doi.org/10.1038/s41592-021-01264-7
Zhao E, Stone MR, Ren X, Guenthoer J, Smythe KS, Pulliam T, Williams SR, Uytingco CR, Taylor SEB, Nghiem P, Bielas JH, Gottardo R (2021) Spatial transcriptomics at subspot resolution with BayesSpace. Nat Biotechnol 39:1375–1384. https://doi.org/10.1038/s41587-021-00935-2
Monjo T, Koido M, Nagasawa S, Suzuki Y, Kamatani Y (2022) Efficient prediction of a spatial transcriptomics profile better characterizes breast cancer tissue sections without costly experimentation. Sci Rep 12:4133. https://doi.org/10.1038/s41598-022-07685-4
Wei H, Lu H, Zhao H (2022) Inferring time-lagged causality using the derivative of single-cell expression. Int J Mol Sci 23:3348convo
Torrisi M, Pollastri G, Le Q (2020) Deep learning methods in protein structure prediction. Comput Struct Biotechnol J 18:1301–1310. https://doi.org/10.1016/j.csbj.2019.12.011
Zhang Y, Qiu L, Ren Y et al (2022) A meta-learning approach to improving radiation response prediction in cancers. Comput Biol Med 150:106163. https://doi.org/10.1016/j.compbiomed.2022.106163
He B, Bergenstråhle L, Stenbeck L, Abid A, Andersson A, Borg Å, Maaskola J, Lundeberg J, Zou J (2020) Integrating spatial gene expression and breast tumour morphology via deep learning. Nat Biomed Eng 4:827–834. https://doi.org/10.1038/s41551-020-0578-x
Schmauch B, Romagnoni A, Pronier E, Saillard C, Maillé P, Calderaro J, Kamoun A, Sefta M, Toldo S, Zaslavskiy M, Clozel T, Moarii M, Courtiol P, Wainrib G (2020) A deep learning model to predict RNA-Seq expression of tumours from whole slide images. Nat Commun 11:3877. https://doi.org/10.1038/s41467-020-17678-4
Chelebian E, Avenel C, Kartasalo K, Marklund M, Tanoglidi A, Mirtti T, Colling R, Erickson A, Lamb AD, Lundeberg J, Wählby C (2021) Morphological features extracted by ai associated with spatial transcriptomics in prostate cancer. Cancers 13:4837. https://doi.org/10.3390/cancers13194837
Hong R, Liu W, DeLair D, Razavian N, Fenyö D (2021) Predicting endometrial cancer subtypes and molecular features from histopathology images using multi-resolution deep learning models. Cell Rep Med 2:100400. https://doi.org/10.1016/j.xcrm.2021.100400
Wang Y, Kartasalo K, Weitz P, Ács B, Valkonen M, Larsson C, Ruusuvuori P, Hartman J, Rantalainen M (2021) Predicting molecular phenotypes from histopathology images: a transcriptome-wide expression-morphology analysis in breast cancer. Can Res 81:5115–5126. https://doi.org/10.1158/0008-5472.CAN-21-0482
Stenbeck L, Bergenstråhle L, Lundeberg J, Borg Å (2021) Human breast cancer in situ capturing transcriptomics. Mendeley Data, V5. https://doi.org/10.17632/29ntw7sh4r.5
Wang Z, Chen J, Hoi SCH (2021) Deep learning for image super-resolution: a survey. IEEE Trans Pattern Anal Mach Intell 43:3365–3387. https://doi.org/10.1109/TPAMI.2020.2982166
Amgad M, Elfandy H, Hussein H, Atteya LA, Elsebaie MAT, Abo Elnasr LS, Sakr RA, Salem HSE, Ismail AF, Saad AM, Ahmed J, Elsebaie MAT, Rahman M, Ruhban IA, Elgazar NM, Alagha Y, Osman MH, Alhusseiny AM, Khalaf MM, Younes A-AF, Abdulkarim A, Younes DM, Gadallah AM, Elkashash AM, Fala SY, Zaki BM, Beezley J, Chittajallu DR, Manthey D, Gutman DA, Cooper LAD (2019) Structured crowdsourcing enables convolutional segmentation of histology images. Bioinformatics 35:3461–3467. https://doi.org/10.1093/bioinformatics/btz083
Huang G, Liu Z, Van Der Maaten L, Weinberger KQ (2017) Densely connected convolutional networks. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, Honolulu, HI, pp 2261–2269. https://doi.org/10.1109/CVPR.2017.243
Zhang Y-D, Satapathy SC, Zhang X, Wang S-H (2021) COVID-19 diagnosis via DenseNet and optimization of transfer learning setting. Cogn Comput. https://doi.org/10.1007/s12559-020-09776-8
Ezzat D, Hassanien AE, Ella HA (2021) An optimized deep learning architecture for the diagnosis of COVID-19 disease based on gravitational search optimization. Appl Soft Comput 98:106742. https://doi.org/10.1016/j.asoc.2020.106742
Jung Y, Kim T, Han M-R, Kim S, Kim G, Lee S, Choi YJ (2022) Ovarian tumor diagnosis using deep convolutional neural networks and a denoising convolutional autoencoder. Sci Rep 12:17024. https://doi.org/10.1038/s41598-022-20653-2
Riasatian A, Babaie M, Maleki D, Kalra S, Valipour M, Hemati S, Zaveri M, Safarpoor A, Shafiei S, Afshari M, Rasoolijaberi M, Sikaroudi M, Adnan M, Shah S, Choi C, Damaskinos S, Campbell CJ, Diamandis P, Pantanowitz L, Kashani H, Ghodsi A, Tizhoosh HR (2021) Fine-tuning and training of densenet for histopathology image representation using TCGA diagnostic slides. Med Image Anal 70:102032. https://doi.org/10.1016/j.media.2021.102032
Deng J, Dong W, Socher R, Li L-J, Kai Li, Li Fei-Fei (2009) ImageNet: a large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, Miami, FL, pp 248–255. https://doi.org/10.1109/CVPR.2009.5206848
Mason A, Rioux J, Clarke SE, Costa A, Schmidt M, Keough V, Huynh T, Beyea S (2020) Comparison of objective image quality metrics to expert radiologists’ scoring of diagnostic quality of MR images. IEEE Trans Med Imaging 39:1064–1072. https://doi.org/10.1109/TMI.2019.2930338
Nilsson J, Akenine-Moller T (2021) Understanding SSL. arXiv 313–330. https://doi.org/10.1201/noe0849385858-25
Buhmann MD (2000) Radial basis functions. Acta Numer 9:1–38. https://doi.org/10.1017/S0962492900000015
Qiu D, Zheng L, Zhu J, Huang D (2021) Multiple improved residual networks for medical image super-resolution. Futur Gener Comput Syst 116:200–208. https://doi.org/10.1016/j.future.2020.11.001
Cavoretto R, De Rossi A, Mukhametzhanov MS, YaD S (2021) On the search of the shape parameter in radial basis functions using univariate global optimization methods. J Glob Optim 79:305–327. https://doi.org/10.1007/s10898-019-00853-3
Gruslova A, McClellan B, Balinda HU, Viswanadhapalli S, Alers V, Sareddy GR, Huang T, Garcia M, deGraffenried L, Vadlamudi RK, Brenner AJ (2021) FASN inhibition as a potential treatment for endocrine-resistant breast cancer. Breast Cancer Res Treat 187:375–386. https://doi.org/10.1007/s10549-021-06231-6
Chaturvedi S, Biswas M, Sadhukhan S, Sonawane A (2023) Role of EGFR and FASN in breast cancer progression. J Cell Commun Signal. https://doi.org/10.1007/s12079-023-00771-w
Wang C, Wang Z, Yao T, Zhou J, Wang Z (2022) The immune-related role of beta-2-microglobulin in melanoma. Front Oncol 12:944722. https://doi.org/10.3389/fonc.2022.944722
Schaafsma E, Fugle CM, Wang X, Cheng C (2021) Pan-cancer association of HLA gene expression with cancer prognosis and immunotherapy efficacy. Br J Cancer 125:422–432. https://doi.org/10.1038/s41416-021-01400-2
Wang C, Lv J, Xue C, Li J, Liu Y, Xu D, Jiang Y, Jiang S, Zhu M, Yang Y, Zhang S (2022) Novel role of COX6c in the regulation of oxidative phosphorylation and diseases. Cell Death Discov 8:1–8. https://doi.org/10.1038/s41420-022-01130-1
Sun X, Li K, Hase M, Zha R, Feng Y, Li B-Y, Yokota H (2022) Suppression of breast cancer-associated bone loss with osteoblast proteomes via Hsp90ab1/moesin-mediated inhibition of TGFβ/FN1/CD44 signaling. Theranostics 12:929–943. https://doi.org/10.7150/thno.66148
Fu Y, Jung AW, Torne RV, Gonzalez S, Vöhringer H, Shmatko A, Yates LR, Jimenez-Linan M, Moore L, Gerstung M (2020) Pan-cancer computational histopathology reveals mutations, tumor composition and prognosis. Nat Cancer 1:800–810. https://doi.org/10.1038/s43018-020-0085-8
Schrammen PL, Ghaffari Laleh N, Echle A, Truhn D, Schulz V, Brinker TJ, Brenner H, Chang-Claude J, Alwers E, Brobeil A, Kloor M, Heij LR, Jäger D, Trautwein C, Grabsch HI, Quirke P, West NP, Hoffmeister M, Kather JN (2022) Weakly supervised annotation-free cancer detection and prediction of genotype in routine histopathology. J Pathol 256:50–60. https://doi.org/10.1002/path.5800
Acknowledgements
The computations in this paper were run on the \(\pi\) 2.0 cluster supported by the Center for High-Performance Computing at Shanghai Jiao Tong University.
Funding
This work was supported by the Neil Shen's SJTU Medical Research Fund (to YK, HL), SJTU Trans-med Awards Research (STAR) 20210106 (to HL), Innovative Research Team of High-Level Local Universities in Shanghai (SHSMU-ZDCX20212200, to HL).
Author information
Authors and Affiliations
Contributions
S. W. participated in the data acquisition, performed the model construction and the statistical analysis, and drafted the manuscript. X. C. Z. participated in the study design and was involved in interpreting study findings and implications. Y. K. participated in its design and drafted the manuscript. H.L. supervised all aspects of the study. All authors read and approved the final manuscript.
Corresponding author
Ethics declarations
Ethics approval and consent to participate
Not applicable.
Consent for publication
Not applicable.
Competing interests
The authors declare that the research was conducted without any commercial or financial relationships that could be construed as a potential conflict of interest.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Key Messages
1. This study proposes a deep learning-based, plug-and-play method for enhancing spot resolution in spatially resolved transcriptomics (SRT) data using a convolutional neural network (CNN) model and shift-predict operation.
2. The method achieves 9 × superresolution on a human breast cancer SRT dataset and outperforms traditional superresolution techniques.
3. The effectiveness of the proposed method is demonstrated using an external dataset and a simulated dataset, providing deeper insights into gene expression patterns and cellular structures.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Wang, S., Zhou, X., Kong, Y. et al. Superresolved spatial transcriptomics transferred from a histological context. Appl Intell 53, 31033–31045 (2023). https://doi.org/10.1007/s10489-023-05190-3
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10489-023-05190-3