Probability Density Function for Clustering Validation.

The choice of the number of clusters is a leading problem in Machine Learning. Validation methods provide solutions, with the drawback that inference is not possible. In this manuscript, we derive a distribution for the number of clusters for clustering validation. The starting point of our approach is the data transformation to the probabilistic space. Then, the dependence of the non-negative factorization to the dimensionality of the space span provides a sequence of the traces when the dimensionality varies. Its limit is a gamma. This result allows a non-excluding discussion when interpreting probabilities as credibility levels, and we open the door to inference for clustering validation.