Detecting entropy increase in categorical data using maximum entropy distribution approximations

We propose a statistical monitoring method to detect the increase of entropy in categorical data. First, we propose a distribution estimationmethod to approximate the probability distribution of the observed categorical data. The problem is formulated as a convex optimization problem, which involves finding the distribution that maximizes Shannon's entropy with the constraint defined by the given confidence intervals on possible distributions. Then we use this procedure to estimate the non-parametric, maximum entropy distribution of an observed data sample and use it for statistical monitoring based on a chi(2)-test statistic. This monitoring scheme was found to be effective in detecting entropy increases in the observed data based on various numerical studies and a real-world case study.