Revisiting Cosine Similarity via Normalized ICA-transformed Embeddings
arxiv(2024)
摘要
Cosine similarity is widely used to measure the similarity between two
embeddings, while interpretations based on angle and correlation coefficient
are common. In this study, we focus on the interpretable axes of embeddings
transformed by Independent Component Analysis (ICA), and propose a novel
interpretation of cosine similarity as the sum of semantic similarities over
axes. To investigate this, we first show experimentally that unnormalized
embeddings contain norm-derived artifacts. We then demonstrate that normalized
ICA-transformed embeddings exhibit sparsity, with a few large values in each
axis and across embeddings, thereby enhancing interpretability by delineating
clear semantic contributions. Finally, to validate our interpretation, we
perform retrieval experiments using ideal embeddings with and without specific
semantic components.
更多查看译文
AI 理解论文
溯源树
样例
![](https://originalfileserver.aminer.cn/sys/aminer/pubs/mrt_preview.jpeg)
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要