A deep-learning framework to foretell most cancers therapy response from histopathology photos by means of imputed transcriptomics


  • Golub, T. R. et al. Molecular classification of most cancers: class discovery and sophistication prediction by gene expression monitoring. Science 286, 531–537 (1999).

    Article 
    CAS 
    PubMed 

    Google Scholar
     

  • Curtis, C. et al. The genomic and transcriptomic structure of two,000 breast tumours reveals novel subgroups. Nature 486, 346–352 (2012).

    Article 
    CAS 
    PubMed 
    PubMed Central 

    Google Scholar
     

  • Doroshow, D. B. & Doroshow, J. H. Genomics and the historical past of precision oncology. Surg. Oncol. Clin. N. Am. 29, 35–49 (2020).

    Article 
    PubMed 

    Google Scholar
     

  • Rosenthal, J. et al. Constructing instruments for machine studying and synthetic intelligence in most cancers analysis: greatest practices and a case examine with the PathML toolkit for computational pathology. Mol. Most cancers Res. 20, 202–206 (2022).

    Article 
    CAS 
    PubMed 

    Google Scholar
     

  • Ström, P. et al. Synthetic intelligence for analysis and grading of prostate most cancers in biopsies: a population-based, diagnostic examine. Lancet Oncol. 21, 222–232 (2020).

    Article 
    PubMed 

    Google Scholar
     

  • Fu, Y. et al. Pan-cancer computational histopathology reveals mutations, tumor composition and prognosis. Nat. Most cancers 1, 800–810 (2020).

    Article 
    CAS 
    PubMed 

    Google Scholar
     

  • Noorbakhsh, J. et al. Deep learning-based cross-classifications reveal conserved spatial behaviors inside tumor histological photos. Nat. Commun. 11, 6367 (2020).

    Article 
    CAS 
    PubMed 
    PubMed Central 

    Google Scholar
     

  • Kim, J. et al. Unsupervised discovery of tissue structure in multiplexed imaging. Nat. Strategies 19, 1653–1661 (2022).

    Article 
    CAS 
    PubMed 
    PubMed Central 

    Google Scholar
     

  • Coudray, N. et al. Classification and mutation prediction from non-small cell lung most cancers histopathology photos utilizing deep studying. Nat. Med. 24, 1559–1567 (2018).

    Article 
    CAS 
    PubMed 
    PubMed Central 

    Google Scholar
     

  • Swiderska-Chadaj, Z. et al. Studying to detect lymphocytes in immunohistochemistry with deep studying. Med. Picture Anal. 58, 101547 (2019).

    Article 
    PubMed 

    Google Scholar
     

  • Yu, Okay.-H. et al. Classifying non-small cell lung most cancers varieties and transcriptomic subtypes utilizing convolutional neural networks. J. Am. Med. Inform. Assoc. 27, 757–769 (2020).

    Article 
    PubMed 
    PubMed Central 

    Google Scholar
     

  • Couture, H. D. et al. Picture evaluation with deep studying to foretell breast most cancers grade, ER standing, histologic subtype, and intrinsic subtype. NPJ Breast Most cancers 4, 30 (2018).

    Article 
    PubMed 
    PubMed Central 

    Google Scholar
     

  • Campanella, G. et al. Medical-grade computational pathology utilizing weakly supervised deep studying on complete slide photos. Nat. Med. 25, 1301–1309 (2019).

    Article 
    CAS 
    PubMed 
    PubMed Central 

    Google Scholar
     

  • Xu, H. et al. Spatial heterogeneity and group of tumor mutation burden with immune infiltrates inside tumors based mostly on complete slide photos correlated with affected person survival in bladder most cancers. J. Pathol. Inform. 13, 100105 (2022).

    Article 
    PubMed 
    PubMed Central 

    Google Scholar
     

  • Kather, J. N. et al. Pan-cancer image-based detection of clinically actionable genetic alterations. Nat. Most cancers 1, 789–799 (2020).

    Article 
    CAS 
    PubMed 
    PubMed Central 

    Google Scholar
     

  • Qu, H. et al. Genetic mutation and organic pathway prediction based mostly on complete slide photos in breast carcinoma utilizing deep studying. NPJ Summary. Oncol. 5, 87 (2021).

    Article 
    CAS 
    PubMed 
    PubMed Central 

    Google Scholar
     

  • Schaumberg, A. J., Rubin, M. A. & Fuchs, T. J. H&E-stained complete slide picture deep studying predicts SPOP mutation state in prostate most cancers. Preprint at bioRxiv https://doi.org/10.1101/064279 (2018).

  • Tsou, P. & Wu, C.-J. Mapping driver mutations to histopathological subtypes in papillary thyroid carcinoma: making use of a deep convolutional neural community. J. Clin. Med. 8, 1675 (2019).

    Article 
    CAS 
    PubMed 
    PubMed Central 

    Google Scholar
     

  • Chang, P. et al. Deep-learning convolutional neural networks precisely classify genetic mutations in gliomas. AJNR Am. J. Neuroradiol. 39, 1201–1207 (2018).

    Article 
    CAS 
    PubMed 
    PubMed Central 

    Google Scholar
     

  • Kather, J. N. et al. Deep studying can predict microsatellite instability instantly from histology in gastrointestinal most cancers. Nat. Med. 25, 1054–1056 (2019).

    Article 
    CAS 
    PubMed 
    PubMed Central 

    Google Scholar
     

  • Kim, R. H. et al. A deep studying strategy for fast mutational screening in melanoma. Preprint at bioRxiv https://doi.org/10.1101/610311 (2019).

  • Chen, M. et al. Classification and mutation prediction based mostly on histopathology H&E photos in liver most cancers utilizing deep studying. NPJ Summary. Oncol. 4, 14 (2020).

    Article 
    CAS 
    PubMed 
    PubMed Central 

    Google Scholar
     

  • Ghaffari Laleh, N. et al. Benchmarking weakly-supervised deep studying pipelines for complete slide classification in computational pathology. Med. Picture Anal. 79, 102474 (2022).

    Article 
    PubMed 

    Google Scholar
     

  • Mobadersany, P. et al. Predicting most cancers outcomes from histology and genomics utilizing convolutional networks. Proc. Natl Acad. Sci. USA 115, E2970–E2979 (2018).

    Article 
    CAS 
    PubMed 
    PubMed Central 

    Google Scholar
     

  • Cheng, J. et al. Integrative evaluation of histopathological photos and genomic information predicts clear cell renal cell carcinoma prognosis. Most cancers Res. 77, e91–e100 (2017).

    Article 
    CAS 
    PubMed 
    PubMed Central 

    Google Scholar
     

  • Beck, A. H. et al. Systematic evaluation of breast most cancers morphology uncovers stromal options related to survival. Sci. Transl. Med. 3, 108ra113 (2011).

    Article 
    PubMed 

    Google Scholar
     

  • Bulten, W. et al. Automated deep-learning system for Gleason grading of prostate most cancers utilizing biopsies: a diagnostic examine. Lancet Oncol. 21, 233–241 (2020).

    Article 
    PubMed 

    Google Scholar
     

  • Courtiol, P. et al. Deep learning-based classification of mesothelioma improves prediction of affected person end result. Nat. Med. 25, 1519–1525 (2019).

    Article 
    CAS 
    PubMed 

    Google Scholar
     

  • Boehm, Okay. M. et al. Multimodal information integration utilizing machine studying improves danger stratification of high-grade serous ovarian most cancers. Nat. Most cancers 3, 723–733 (2022).

    Article 
    CAS 
    PubMed 
    PubMed Central 

    Google Scholar
     

  • Chen, Y. et al. Computational pathology improves danger stratification of a multi-gene assay for early stage ER+ breast most cancers. NPJ Breast Most cancers 9, 40 (2023).

    Article 
    CAS 
    PubMed 
    PubMed Central 

    Google Scholar
     

  • Zheng, H., Momeni, A., Cedoz, P.-L., Vogel, H. & Gevaert, O. Complete slide photos replicate DNA methylation patterns of human tumors. NPJ Genom. Med. 5, 11 (2020).

    Article 
    CAS 
    PubMed 
    PubMed Central 

    Google Scholar
     

  • Wang, H. et al. Mitosis detection in breast most cancers pathology photos by combining handcrafted and convolutional neural community options. J. Med. Imaging (Bellingham) 1, 034003 (2014).

    Article 
    PubMed 

    Google Scholar
     

  • Turkki, R., Linder, N., Kovanen, P. E., Pellinen, T. & Lundin, J. Antibody-supervised deep studying for quantification of tumor-infiltrating immune cells in hematoxylin and eosin stained breast most cancers samples. J. Pathol. Inform. 7, 38 (2016).

    Article 
    PubMed 
    PubMed Central 

    Google Scholar
     

  • Saltz, J. et al. Spatial group and molecular correlation of tumor-infiltrating lymphocytes utilizing deep studying on pathology photos. Cell Rep. 23, 181–193 (2018).

    Article 
    CAS 
    PubMed 
    PubMed Central 

    Google Scholar
     

  • Web page, D. B. et al. Spatial analyses of immune cell infiltration in most cancers: present strategies and future instructions: a report of the Worldwide Immuno-Oncology Biomarker Working Group on Breast Most cancers. J. Pathol. 260, 514–532 (2023).

    Article 
    CAS 
    PubMed 

    Google Scholar
     

  • Wang, X. et al. Spatial interaction patterns of most cancers nuclei and tumor-infiltrating lymphocytes (TILs) predict scientific profit for immune checkpoint inhibitors. Sci. Adv. 8, eabn3966 (2022).

    Article 
    CAS 
    PubMed 
    PubMed Central 

    Google Scholar
     

  • Hu, J. et al. Utilizing deep studying to foretell anti-PD-1 response in melanoma and lung most cancers sufferers from histopathology photos. Transl. Oncol. 14, 100921 (2021).

    Article 
    CAS 
    PubMed 

    Google Scholar
     

  • Johannet, P. et al. Utilizing machine studying algorithms to foretell immunotherapy response in sufferers with superior melanoma. Clin. Most cancers Res. 27, 131–140 (2021).

    Article 
    CAS 
    PubMed 

    Google Scholar
     

  • Zhang, F. et al. Predicting therapy response to neoadjuvant chemoradiotherapy in native superior rectal most cancers by biopsy digital pathology picture options. Clin. Transl. Med. 10, e110 (2020).

    Article 
    PubMed 
    PubMed Central 

    Google Scholar
     

  • Honomichl, N. HER2 and trastuzumab therapy response H&E slides with tumor ROI annotations (HER2 tumor ROIs). The Most cancers Imaging Archive https://doi.org/10.7937/E65C-AM96 (2022).

  • He, B. et al. Integrating spatial gene expression and breast tumour morphology by way of deep studying. Nat. Biomed. Eng. 4, 827–834 (2020).

    Article 
    CAS 
    PubMed 

    Google Scholar
     

  • Wang, Y. et al. Predicting molecular phenotypes from histopathology photos: a transcriptome-wide expression–morphology evaluation in breast most cancers. Most cancers Res. 81, 5115–5126 (2021).

    Article 
    CAS 
    PubMed 
    PubMed Central 

    Google Scholar
     

  • Monjo, T., Koido, M., Nagasawa, S., Suzuki, Y. & Kamatani, Y. Environment friendly prediction of a spatial transcriptomics profile higher characterizes breast most cancers tissue sections with out pricey experimentation. Sci. Rep. 12, 4133 (2022).

    Article 
    CAS 
    PubMed 
    PubMed Central 

    Google Scholar
     

  • Schmauch, B. et al. A deep studying mannequin to foretell RNA-Seq expression of tumours from complete slide photos. Nat. Commun. 11, 3877 (2020).

    Article 
    CAS 
    PubMed 
    PubMed Central 

    Google Scholar
     

  • Levy-Jurgenson, A., Tekpli, X., Kristensen, V. N. & Yakhini, Z. Spatial transcriptomics inferred from pathology whole-slide photos hyperlinks tumor heterogeneity to survival in breast and lung most cancers. Sci. Rep. 10, 18802 (2020).

    Article 
    CAS 
    PubMed 
    PubMed Central 

    Google Scholar
     

  • Alsaafin, A., Safarpoor, A., Sikaroudi, M., Hipp, J. D. & Tizhoosh, H. R. Studying to foretell RNA sequence expressions from complete slide photos with functions for search and classification. Commun. Biol. 6, 304 (2023).

    Article 
    CAS 
    PubMed 
    PubMed Central 

    Google Scholar
     

  • Dinstag, G. et al. Clinically oriented prediction of affected person response to focused and immunotherapies from the tumor transcriptome. Med 4, 15–30 (2023).

    Article 
    CAS 
    PubMed 

    Google Scholar
     

  • Sammut, S.-J. et al. Multi-omic machine studying predictor of breast most cancers remedy response. Nature 601, 623–629 (2022).

    Article 
    CAS 
    PubMed 

    Google Scholar
     

  • Zheng, Y. et al. Digital profiling of most cancers transcriptomes from histology photos with grouped imaginative and prescient consideration. Preprint at bioRxiv https://doi.org/10.1101/2023.09.28.560068 (2023).

  • Hanahan, D. & Weinberg, R. A. Hallmarks of most cancers: the subsequent technology. Cell 144, 646–674 (2011).

    Article 
    CAS 
    PubMed 

    Google Scholar
     

  • Iorio, F. et al. Pathway-based dissection of the genomic heterogeneity of most cancers hallmarks’ acquisition with SLAPenrich. Sci. Rep. 8, 6713 (2018).

    Article 
    PubMed 
    PubMed Central 

    Google Scholar
     

  • Subramanian, A. et al. Gene set enrichment evaluation: a knowledge-based strategy for decoding genome-wide expression profiles. Proc. Natl Acad. Sci. USA 102, 15545–15550 (2005).

    Article 
    CAS 
    PubMed 
    PubMed Central 

    Google Scholar
     

  • Richardsen, E. et al. Analysis of the proliferation marker Ki-67 in a big prostatectomy cohort. PLoS ONE 12, e0186852 (2017).

    Article 
    PubMed 
    PubMed Central 

    Google Scholar
     

  • Whitfield, M. L., George, L. Okay., Grant, G. D. & Perou, C. M. Widespread markers of proliferation. Nat. Rev. Most cancers 6, 99–106 (2006).

    Article 
    CAS 
    PubMed 

    Google Scholar
     

  • Liberzon, A. et al. The Molecular Signatures Database (MSigDB) hallmark gene set assortment. Cell Syst. 1, 417–425 (2015).

    Article 
    CAS 
    PubMed 
    PubMed Central 

    Google Scholar
     

  • Farahmand, S. et al. Deep studying skilled on hematoxylin and eosin tumor area of curiosity predicts HER2 standing and trastuzumab therapy response in HER2+ breast most cancers. Mod. Pathol. 35, 44–51 (2022).

    Article 
    CAS 
    PubMed 

    Google Scholar
     

  • Strauss, J. et al. Bintrafusp alfa, a bifunctional fusion protein focusing on TGF-β and PD-L1, in sufferers with human papillomavirus-associated malignancies. J. Immunother. Most cancers 8, e001395 (2020).

    Article 
    PubMed 
    PubMed Central 

    Google Scholar
     

  • Kanopoulos, N., Vasanthavada, N. & Baker, R. L. Design of a picture edge detection filter utilizing the Sobel operator. IEEE J. Strong-State Circuits 23, 358–367 (1988).

    Article 

    Google Scholar
     

  • Macenko, M. et al. A technique for normalizing histology slides for quantitative evaluation. in 2009 IEEE Worldwide Symposium on Biomedical Imaging: From Nano to Macro 1107–1110 (IEEE, 2009); https://doi.org/10.1109/ISBI.2009.5193250

  • He, Okay., Zhang, X., Ren, S. & Solar, J. Deep residual studying for picture recognition. in 2016 IEEE Convention on Pc Imaginative and prescient and Sample Recognition (CVPR) 770–778 (IEEE, 2016); https://doi.org/10.1109/cvpr.2016.90

  • Hoang, D.-T. et al. DeepPT: a deep studying mannequin for predicting transcriptomics from histopathology photos. Zenodo https://doi.org/10.5281/zenodo.11125591 (2024).

  • Hot Topics

    Related Articles