My current research focuses on using Genetic Algorithms to generate
visualizations of images at the decision boundary of CNN classifiers. This project
is carried out under the guidance of Prof. Jun Sun
at the Singapore Management University.
Video Colorization Dataset & Benchmark
Compared to its peer, image colorization, video colorization is a
relatively unexplored area in computer vision. Most of the models
available for video colorization are extensions of image colorization,
and hence are unable to address some unique issues in video domain.
In this project, we evaluated the applicability of image colorization
techniques for video colorization, identifying problems inherent to
videos and attributes affecting them. We developed a dataset and
benchmark to measure the effect of such attributes to
video colorization quality and demonstrate how our benchmark aligns
with human evaluations.
FlowChroma - A Deep Recurrent Network for Video Colorization
We developed an automated video colorization framework that minimizes the
flickering of colors across frames. If we apply image colorization
techniques to successive frames of a video, they treat each frame as a
separate colorization task. Thus, they do not necessarily maintain the
colors of a scene consistently across subsequent frames.
The proposed solution includes a novel deep recurrent encoder-decoder
architecture which is capable of maintaining temporal and contextual
coherence between consecutive frames of a video. We use a high-level
semantic feature extractor to automatically identify the context of a
scenario, with a custom fusion layer that combines the spatial and
temporal features of a frame sequence.