Dispatched routing networks
user-5ebe28d54c775eda72abcdf7(2019)
Abstract
Routing and Recursive Routing Networks (RRNs) are highly expressive neural networks with modular self-assembling architectures that have proven successful in complex NLU tasks. However, this expressive power can at times be both a blessing and a curse. For many practical problems, vanilla RRNs tend to overfit to the limited data available. However, recent work has shown that high-quality meta-information can be extremely useful as a guide for routing in settings that require sample efficiency. Unfortunately, this meta-information is highly problem-dependent, and oftentimes it is not available at test-time. To compensate, we introduce an additional network that is trained jointly with the routing network to map samples to meta-information: a dispatcher. The dispatcher’s goal is finding groups of samples that are as useful to the router as the groups defined by meta-information. We find that RRNs augmented with an end-to-end dispatcher achieve strong performance in multi-task learning scenarios while exhibiting high levels of generalization when adapting to new, unseen tasks.
MoreTranslated text
AI Read Science
Must-Reading Tree
Example
![](https://originalfileserver.aminer.cn/sys/aminer/pubs/mrt_preview.jpeg)
Generate MRT to find the research sequence of this paper
Chat Paper
Summary is being generated by the instructions you defined