An Introduction to Local Entropy and Local Unicity

As introduced by Shannon in “Communication Theory of Secrecy Systems”, entropy and unicity distance are defined at a global level, under the assumption that the properties of symbols resemble that of independent random variables. However, while applying entropy and unicity to language(s), e.g., encryption and decryption, the symbols (letters) of a language are not independent. Thus, we introduce a new measure, $H_{L}(s)$, called the “local entropy” of a string s. $H_{L}(s)$ includes a priori information about the language and text at the time of application. Since the unicity distance is dependent on the entropy (because entropy is the basis of calculations for the unicity distance), local entropy leads to a local unicity distance for a string. Our local entropy measure explains why some texts are susceptible to decryption using fewer symbols than predicted by Shannon’s unicity while other texts require more. We demonstrate local entropy using a substitution cipher along with the results for an algorithm based on the principle, and show that Shannon’s unicity is an average measure rather than a lower bound; this motivates us to present a discussion on the implications of local entropy and unicity distance.