Tracking Emerges by Colorizing Videos

• Google Research, 2018 ### Main Strategy
• Use large amounts of unlabeled video to learn models for visual tracking
  • Without manual human supervision.
• Colorization
  • We leverage the natural temporal coherency of color to create a model that learns to colorize gray-scale videos by copying colors from a reference frame.
  • Quantitative and qualitative experiments suggest that this task causes the model to automatically learn to track visual regions.
• Outperform Optical flow
  • Although the model is trained without any ground-truth labels, our method learns to track well enough to outperform optical flow based methods.
• Advancing Colorization
  • Our results suggest that failures to track are correlated with failures to colorize, indicating that advancing video colorization may further improve self-supervised visual tracking.