Preference-Based Planning in Stochastic Environments: From Partially-Ordered Temporal Goals to Most Preferred Policies
CoRR(2024)
摘要
Human preferences are not always represented via complete linear orders: It
is natural to employ partially-ordered preferences for expressing incomparable
outcomes. In this work, we consider decision-making and probabilistic planning
in stochastic systems modeled as Markov decision processes (MDPs), given a
partially ordered preference over a set of temporally extended goals.
Specifically, each temporally extended goal is expressed using a formula in
Linear Temporal Logic on Finite Traces (LTL_f). To plan with the partially
ordered preference, we introduce order theory to map a preference over temporal
goals to a preference over policies for the MDP. Accordingly, a most preferred
policy under a stochastic ordering induces a stochastic nondominated
probability distribution over the finite paths in the MDP. To synthesize a most
preferred policy, our technical approach includes two key steps. In the first
step, we develop a procedure to transform a partially ordered preference over
temporal goals into a computational model, called preference automaton, which
is a semi-automaton with a partial order over acceptance conditions. In the
second step, we prove that finding a most preferred policy is equivalent to
computing a Pareto-optimal policy in a multi-objective MDP that is constructed
from the original MDP, the preference automaton, and the chosen stochastic
ordering relation. Throughout the paper, we employ running examples to
illustrate the proposed preference specification and solution approaches. We
demonstrate the efficacy of our algorithm using these examples, providing
detailed analysis, and then discuss several potential future directions.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要