Personality-aware Human-centric Multimodal Reasoning: A New Task, Dataset and Baselines
arxiv(2023)
摘要
Personality traits, emotions, and beliefs shape individuals' behavioral
choices and decision-making processes. However, for one thing, the affective
computing community normally focused on predicting personality traits but
overlooks their application in behavior prediction. For another, the multimodal
reasoning task emphasized the prediction of future states and behaviors but
often neglected the incorporation of individual personality traits. In this
work, we introduce a new task called Personality-aware Human-centric Multimodal
Reasoning (PHMR) (T1), with the goal of forecasting the future behavior of a
particular individual using multimodal information from past instances, while
integrating personality factors. We accordingly construct a new dataset based
on six television shows, encompassing 225 characters and 12k samples. To
establish a benchmark for the task, we propose seven baseline methods: three
adapted from related tasks, two pre-trained model, and two multimodal large
language models. The experimental results demonstrate that incorporating
personality traits enhances human-centric multimodal reasoning performance. To
further solve the lack of personality annotation in real-life scenes, we
introduce an extension task called Personality-predicted Human-centric
Multimodal Reasoning task (T2) along with the corresponding dataset and method.
We will make our dataset and code available on GitHub.
更多查看译文
关键词
reasoning,personality-aware,human-centric
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要