Gene-MOE: A Sparsely-gated Framework for Pan-Cancer Genomic Analysis
CoRR(2023)
摘要
Analyzing the genomic information from the Pan-Cancer database can help us
understand cancer-related factors and contribute to the cancer diagnosis and
prognosis. However, existing computational methods and deep learning methods
can not effectively find the deep correlations between tens of thousands of
genes, which leads to precision loss. In this paper, we proposed a novel
pretrained model called Gene-MOE to learn the general feature representations
of the Pan-Cancer dataset and transfer the pretrained weights to the downstream
tasks. The Gene-MOE fully exploits the mixture of expert (MOE) layers to learn
rich feature representations of high-dimensional genes. At the same time, we
build a mixture of attention expert (MOAE) model to learn the deep semantic
relationships within genetic features. Finally, we proposed a new
self-supervised pretraining strategy including loss function design, data
enhancement, and optimization strategy to train the Gene-MOE and further
improve the performance for the downstream analysis. We carried out cancer
classification and survival analysis experiments based on the Gene-MOE.
According to the survival analysis results on 14 cancer types, using Gene-MOE
outperformed state-of-the-art models on 12 cancer types. According to the
classification results, the total accuracy of the classification model for 33
cancer classifications reached 95.2\%. Through detailed feature analysis, we
found the Gene-MOE model can learn rich feature representations of
high-dimensional genes.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要