CNN-based subpixel rendering technique with additional pixel arrangement information for high quality displays

Subpixel rendering approaches are necessary to provide apparently high resolution images when there are difficulties in implementing the display with the target high resolution. This paper presents a subpixel rendering technique based on the convolutional neural network (CNN) for a diamond-shaped PenTile pixel arrangement. The proposed subpixel rendering network (SPRNN) takes the concatenated input of high resolution image and pixel arrangement information for each color component. SPRNNs for three colors share weights, leading to the substantial reduction on the number of weights. The loss function is defined as the squared error between original and virtual images, where virtual images are generated from the down-sampled SPRNN outputs by the human visual system (HVS) model. The proposed SPRNN outperforms other down-sampling algorithms such as DPD, DSD, DDSD, PDAF, DSD-FA, DDSD-FA, and QPD in terms of PSNRz and SSIMz estimated from virtual images as well as SPA metrics measured at the down-sampled images. PSNRz and SSIMz of the proposed SPRNN are measured as 32.6027 dB and 0.9813. In addition, the light version SPRNN (SPRNNL) also achieves high PSNRz and SSIMz of 32.5192 dB and 0.9810 that are bigger than previous down-sampling schemes, while its computational complexity is reduced by around 99.8%, compared to SPRNN.