Kolmogorov–Smirnov Test-Based Actively-Adaptive Thompson Sampling for Non-Stationary Bandits

We consider the nonstationary multiarmed bandit framework and propose a Kolmogorov–Smirnov (KS) test based Thompson sampling (TS) algorithm named TS-KS that actively detects change points and resets the TS parameters once a change is detected. In particular, for the two-armed bandit case, we derive bounds on the number of samples of the reward distribution to detect the chan...