IRNet-RS: image retargeting network via relative saliency

Image retargeting (IR) aims to fit different display terminals by changing the image aspect ratio while retaining its important content. The IR methods utilize a saliency map to predict the importance of pixels; however, existing methods assume that multiple objects within the saliency map have the same importance, which does not mimic the human attention mechanism and thus results in inadequate protection of the main content during the retargeting operation. To solve these problems, we propose an image retargeting network via relative saliency (IRNet-RS), which has three key steps: relative saliency detection, edge detection, and IR operation. By using a relative saliency stratified supervision module and saliency rank guidance for the refinement module, IRNet-RS detects the salient objects in an image, allocates different saliency weights to them to obtain a relative saliency map, and then fuses it with the edge detection map to generate an importance map. An adaptive duplicate convolution module is also designed to help the network learn the pixel-by-pixel shift map from the source to the target grid, followed by retargeting the image using pixel shifting. Our experimental results show that the proposed IRNet-RS achieves better performance than alternative methods in terms of saliency rank prediction and retargeting results.