Finding Visual Task Vectors
arxiv(2024)
摘要
Visual Prompting is a technique for teaching models to perform a visual task
via in-context examples, without any additional training. In this work, we
analyze the activations of MAE-VQGAN, a recent Visual Prompting model, and find
task vectors, activations that encode task-specific information. Equipped with
this insight, we demonstrate that it is possible to identify the task vectors
and use them to guide the network towards performing different tasks without
providing any input-output examples. To find task vectors, we compute the
average intermediate activations per task and use the REINFORCE algorithm to
search for the subset of task vectors. The resulting task vectors guide the
model towards performing a task better than the original model without the need
for input-output examples.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要