Quantifying the benefits of increasing decision-making frequency for health applications with regular decision epochs

Sequential decision-making problems in the context of uncertainty naturally arise in healthcare settings. In general, the frequency at which decisions can be made or changed is determined by physical limitations, such as the frequency of doctor's visits or transplantation offers. Quantifying the benefits of increasing the frequency of decision-making allows us to quantify the value of changing these physical constraints and thus improve the quality of care. In this article, we study the value provided by having additional decision-making opportunities in each epoch. We model this problem using a Markov Decision Process (MDP) framework. We provide structural properties of the optimal policies and quantify the difference in optimal values between MDP problems of different decision-making frequencies. We analyze numerical examples using liver transplantation in high-risk patients and treatment initiation in chronic kidney disease to illustrate our findings.