Compared to its peer, image colorization, video colorization is a relatively unexplored area in computer vision. Most of the models available for video colorization are extensions of image colorization, and hence are unable to address some unique issues in video domain.

In this project, we evaluated the applicability of image colorization techniques for video colorization, identifying problems inherent to videos and attributes affecting them. We developed a dataset and benchmark to measure the effect of such attributes to video colorization quality and demonstrate how our benchmark aligns with human evaluations.

We developed an automated video colorization framework that minimizes the flickering of colors across frames. If we apply image colorization techniques to successive frames of a video, they treat each frame as a separate colorization task. Thus, they do not necessarily maintain the colors of a scene consistently across subsequent frames.

The proposed solution includes a novel deep recurrent encoder-decoder architecture which is capable of maintaining temporal and contextual coherence between consecutive frames of a video. We use a high-level semantic feature extractor to automatically identify the context of a scenario, with a custom fusion layer that combines the spatial and temporal features of a frame sequence.