Design of Multi-Armed Bandit-Based Routing for in-Network Caching

This paper proposes a multi-armed bandit-based routing method for in-network cache-enabled networks. In-network caching enables intermediate routers to store contents in their cache, which is adopted in new networking paradigms such as Information-Centric Networking. Clients can download the cached contents from the routers instead of original content servers. Therefore, applying in-network caching saves network resources and reduces response time of content downloading. In order to efficiently utilize the in-network caching mechanism, the proposed method provides a routing algorithm for content requests based on a multi-armed bandit algorithm, which is one of reinforcement learning. In the proposed method, we consider the routers as players in the multi-armed bandit problem. Each router forwards content requests to appropriate output ports based on rewards calculated by the multi-armed bandit algorithm. Furthermore, the proposed method provides a collaborative caching mechanism where adjacent routers share their caching information with each other. By doing so, the content requests are likely to find routers having target contents, and thus the cache hit ratio is expected to be improved while reducing the average hop count to retrieve the target contents. Through simulation experiments, we show the effectiveness of the proposed routing method.