Perceptually Driven Frame-Rate Manipulations

Rising popularity of high frame-rates (HFR) in the films has sparked a lot of debate. Proponents of the technology claim it to be more realistic, smoother, and sharper. While opponents claim that it looks too realistic and smooth, rendering the films “cheap” and “soap-operatic”. Some critics also complain that HFR looks sped-up. Variable framerate technology gives filmmakers more control on manipulating frame-rates. However, there is no method or guide for the selection of frame-rate. First part of this thesis investigates the urban legend that speed perception is dependent on frame-rate. To measure the dependence of frame-rate on speed perception, we performed three psychophysical experiments using basic stimuli (Gabor patches), animated content and real-world footage. The results show that there is no significant effect of frame-rate on speed perception. There was a lot of variation in participants’ responses. Inconsistencies in people’s opinions regarding HFR content is not new. There are studies which demonstrate the incongruity of participants’ opinions regarding HFR. This thesis concludes that frame-rate does not alter speed perception. Perhaps, lack of artifacts in HFR is some times being misconstrued by some imaginative viewers as difference in speed. Even if there is an effect, it is too small to be measured. Flicker can be defined as jittery motion due to temporal variation in pixels. Since flicker is the most prominent difference between two frame-rates, we propose that it is also a major contributor to the “film-look”. This work explores the relationship between frame-rate and flicker. Given an input video, we present a model to calculate per-pixel flicker. We also present a flicker driven guide to use variable frame-rate technology. For this purpose, we take input video and goal flicker visibility maps, and solve an optimization problem for spatially and temporally variable, and spatially and temporally smooth frame-rate maps, such that perceived flicker matches with the goal flicker. We use these frame-rate maps as an input for the method proposed by Templin et al. to produce output frames, which have required flicker. By manipulating flicker with our technique, filmmakers can reap the benefits of HFR, without incurring the unwanted “soap-operatic” effects.