Learning-based Optimal Admission Control in a Single Server Queuing System

We consider admission control for a first-in first-out single class single server queueing model with Poisson arrivals and exponential service times. Specifically, there is a dispatcher that decides on admitting arrivals with the aim to maximize total net profit - each admitted arrival yields a positive reward $R$ (obtained after customer finishes service) that needs to be balanced by a holding cost for the (homogeneous) customers in the queue. Whereas the capacity of this queue is infinite, the dispatcher may decide to reject any customers joining the queue with the profit objective in mind. When the arrival and service rates are known, this model was studied by Naor [1], but in our investigation the dispatcher is assumed to know the arrival rate but not the service rate.