Attention Based Models for Cell Type Classification on Single-Cell RNA-Seq Data

Cell type classification serves as one of the most fundamental analyses in bioinformatics. It helps recognizing various cells in cancer microenvironment, discovering new cell types and facilitating other downstream tasks. Single-cell RNA-sequencing (scRNA-seq) technology can profile the whole transcriptome of each cell, thus enabling cell type classification. However, high-dimensional scRNA-seq data pose serious challenges on cell type classification. Existing methods either classify the cells with reliance on the prior knowledge or by using neural networks whose massive parameters are hard to interpret. In this paper, we propose two novel attention-based models for cell type classification on single-cell RNA-seq data. The first model, Cell Feature Attention Network (CFAN), captures the features of a cell and performs attention model on them. To further improve interpretation, the second model, Cell-Gene Representation Attention Network (CGRAN), directly concretizes tokens as cells and genes and uses the cell representation renewed by self-attention over the cell and the genes to predict cell type. Both models show excellent performance in cell type classification; additionally, the key genes with high attention weights in CGRAN indicate and identify the marker genes of the cell types, thus proving the model’s biological interpretation.