Estimating Mutual Information via Geodesic kNN

Previous chapter Next chapter Full AccessProceedings Proceedings of the 2022 SIAM International Conference on Data Mining (SDM)Estimating Mutual Information via Geodesic kNNAlexander Marx and Jonas FischerAlexander Marx and Jonas Fischerpp.415 - 423Chapter DOI:https://doi.org/10.1137/1.9781611977172.47PDFBibTexSections ToolsAdd to favoritesExport CitationTrack CitationsEmail SectionsAboutAbstract Estimating mutual information (MI) between two continuous random variables X and Y allows to capture non-linear dependencies between them, non-parametrically. As such, MI estimation lies at the core of many data science applications. Yet, robustly estimating MI for high-dimensional X and Y is still an open research question. In this paper, we formulate this problem through the lens of manifold learning. That is, we leverage the common assumption that the information of X and Y is captured by a low-dimensional manifold embedded in the observed high-dimensional space and transfer it to MI estimation. As an extension to state-of-the-art kNN estimators, we propose to determine the k-nearest neighbors via geodesic distances on this manifold rather than from the ambient space, which allows us to estimate MI even in the high-dimensional setting. An empirical evaluation of our method, G-KSG, against the state-of-the-art shows that it yields good estimations of MI in classical benchmark and manifold tasks, even for high dimensional datasets, which none of the existing methods can provide. Previous chapter Next chapter RelatedDetails Published:2022eISBN:978-1-61197-717-2 https://doi.org/10.1137/1.9781611977172Book Series Name:ProceedingsBook Code:PRDT22Book Pages:1-737