Spatiotemporal Consistency in Rendering and Deep Learning

In computer graphics, a lot of research has been done on the generation of realistic images. The goal is to avoid aliasing along edges, noise as a result of Monte-Carlo-based methods and other artifacts that can arise during rendering. Lately, deep learning has been explored as a tool for solving these problems. These methods have seen some success. What has been shown less attention, however, is the quality of image sequences. If, for example, an artificial neural network is used to denoise an image, without any information about the other images in the sequence, there is a risk that the denoised sequence appears to flicker, or in other ways appear disturbing to the observer, despite the single, denoised image looking good when compared to a reference.

In our research, we explore these problems and try to solve them in different ways. For example, we have investigated the effect of presenting image sequences at a much higher speed (frame rate) than usual (240 FPS compared to 60 FPS) and have seen that this change can have significant, positive effects on how an image sequence is perceived.