Background
Spatial transcriptomics (ST) has greatly advanced gene biomarker discovery1 and understanding of intra-tumoral heterogeneity.2 Deep learning methods applied to H&E images have improved clinical trials,3 inspiring the term H&E 2.0.4 Our work aims to enhance this by applying deep learning to H&E images in ST, reducing costs and enabling retrospective analysis on public H&E datasets without ST labels. Convolutional Neural Network (CNN) based methods like ST-Net5 have shown promise in breast cancer studies. We aim to address two aspects not covered by ST-Net: applicability to other tumors and model generalizability. We have retrained ST-Net on an in-house H&E imaging dataset from a Hepatocellular carcinoma (HCC) patient and tested its gene expression prediction capability on a public dataset, TCGA-LIHC.
Methods
A single HCC sample was profiled using the 10x Visium technique. Image patches of 296×296 pixels were produced from the H&E image, each encompassing a Visium spot (figure 1). The ST-Net5 was trained using the image patches and Visium readouts of the top 250 most prevalent genes in this sample. A randomly selected 20% patches were held-out for validation purposes. The genes CD74 and ALB were selected for further investigation due to their status as HCC biomarkers.6 7 The ST-Net model, trained on HCC data, was validated using the public TCGA liver cancer cohort, specifically focusing on the two samples exhibiting the highest and lowest expression of ALB. To manage computational difficulties, we randomly selected 10% of all the image tiles for model prediction.
Results
In the 20% held-out regions, the model attained an average Spearman’s correlation of 0.34 across the 250 target genes, with correlations of 0.53 for ALB and 0.34 for CD74 (figure 2A). The gene expression map generated by the model was in agreement with the Visium measurements for both ALB and CD74 (figure 2B). Evaluation on TCGA liver cancer cohort shows promising results. A relatively higher%regions were predicted to have increased ALB expression in samples exhibiting the highest bulk RNAseq values as compared to those exhibiting the lowest bulk RNAseq values, 1.34% and 0.323% vs. 0.234% and 0.0187%, respectively (figure 3).
Conclusions
We present an extended application of existing ST prediction mode, ST-Net, in HCC and in public TCGA dataset. Future efforts to improve the model’s performance and generalizability could encompass implementing H&E color normalization and enhancing model explainability, such as through Gradient-weighted Class Activation Mapping (Grad-CAM).
Acknowledgements
This work was supported by the Bioinformatics Institute, Singapore Immunology Network, and Insitute of Molecular and Cell Biology, Agency for Science, Technology and Research (A*STAR)
References
Zhang L, Chen D, Song D, Liu X, Zhang Y, Xu X, Wang X. Clinical and translational values of spatial transcriptomics. Signal Transduct Target Ther. 2022;7(1):1–17. doi: 10.1038/s41392–022-00960-w.
Li Q, Zhang X, Ke R. Spatial transcriptomics for tumor heterogeneity analysis. Front Genet. 2022;13:906158. doi: 10.3389/fgene.2022.906158.
Qaiser T, Lee CY, Vandenberghe M, Yeh J, Gavrielides MA, Hipp J, Scott M, Reischl J. Usability of deep learning and H&E images predict disease outcome-emerging tool to optimize clinical trials. NPJ Precis Oncol. 2022;6(1):1–12. doi: 10.1038/s41698–022-00275–7.
Comiter C, Vaishnav ED, Ciampricotti M, Li B, Yang Y, Rodig SJ, Turner M, Pfaff KL, Jané-Valbuena J, Slyper M, Waldman J, Vigneau S, Wu J, Blosser TR, Segerstolpe Å, Abravanel D, Wagle N, Zhuang X, Rudin CM, Regev A. Inference of single cell profiles from histology stains with the Single-Cell omics from Histology Analysis Framework (SCHAF). bioRxiv. 2023 Mar 21;2023.03.21.533680. doi: 10.1101/2023.03.21.533680.
He B, Bergenstråhle L, Stenbeck L, Abid A, Andersson A, Borg Å, Maaskola J, Lundeberg J, Zou J. Integrating spatial gene expression and breast tumor morphology via deep learning. Nat Biomed Eng. 2020;4(8):827–834. doi: 10.1038/s41551–020-0578-x.
Fu X, Yang Y, Zhang D. Molecular mechanism of albumin in suppressing invasion and metastasis of hepatocellular carcinoma. Liver Int. 2022;42(3):696–709. doi: 10.1111/liv.15115.
Xiao N, Li K, Zhu X, Xu B, Liu X, Lei M, Sun HC. CD74+ macrophages are associated with favorable prognosis and immune contexture in hepatocellular carcinoma. Cancer Immunol Immunother. 2022;71(1):57–69. doi: 10.1007/s00262–021-02962-z.
Abstract 1279 Figure 1
An overview of our proposed study on (A) the extended application of ST-net to liver cancer and (B) its generalizability assessment on public TCGA liver cancer cohort. A resected HCC tissue was profiled using Visium technique, accompanied by a corresponding H&Estained image. The H&E image was tiled and split into an 80/20 train/test sets for re-training and validating the ST-Net. Once the model is trained, a gene expression map can be inferred from unseen image patch input
Abstract 1279 Figure 2
Evaluation of ST-Net performance on the 20% held-out dataset of the HCC sample. (A) This shows the distribution of Spearman’s correlation for the 250 target genes (P < 0.05); each dot on the boxplot represents an individual gene. The highest performing 10 genes are labeled for ease of reference. (B) The gene expression map created by the model (on the left) is compared to the actual Visium measurements (ground truth on the right) for ALB and CD74; individual dots represent model output and Visium spots respectively. The values were center-adjusted to zero and standard normalized to aid visualization
Abstract 1279 Figure 3
Evaluation of model perfomance on public TCGA liver cancer cohort. In the randomly selected 10% regions (image patches), a relatively higher number of regions were predicted to have increased ALB expression (> 0.5) in (top) samples exhibiting the highest bulk RNAseq values as compared to (bottom) samples exhibiting the lowest bulk RNAseq values. Each dot in the figure represents a randomly selected patch. Regions (image patches) with the highest predicted ALB expression are highlighted in yellow and increased in size