Geometric Distortion Guided Transformer for Omnidirectional Image Super-Resolution
CoRR(2024)
摘要
As virtual and augmented reality applications gain popularity,
omnidirectional image (ODI) super-resolution has become increasingly important.
Unlike 2D plain images that are formed on a plane, ODIs are projected onto
spherical surfaces. Applying established image super-resolution methods to
ODIs, therefore, requires performing equirectangular projection (ERP) to map
the ODIs onto a plane. ODI super-resolution needs to take into account
geometric distortion resulting from ERP. However, without considering such
geometric distortion of ERP images, previous deep-learning-based methods only
utilize a limited range of pixels and may easily miss self-similar textures for
reconstruction. In this paper, we introduce a novel Geometric Distortion Guided
Transformer for Omnidirectional image Super-Resolution (GDGT-OSR).
Specifically, a distortion modulated rectangle-window self-attention mechanism,
integrated with deformable self-attention, is proposed to better perceive the
distortion and thus involve more self-similar textures. Distortion modulation
is achieved through a newly devised distortion guidance generator that produces
guidance by exploiting the variability of distortion across latitudes.
Furthermore, we propose a dynamic feature aggregation scheme to adaptively fuse
the features from different self-attention modules. We present extensive
experimental results on public datasets and show that the new GDGT-OSR
outperforms methods in existing literature.
更多查看译文
AI 理解论文
溯源树
样例
![](https://originalfileserver.aminer.cn/sys/aminer/pubs/mrt_preview.jpeg)
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要