Global motion compensation (GMC) removes the impact of camera motion and creates a video in which the background appears static over the progression of time. Various vision problems, such as human activity recognition, background reconstruction, and multi-object tracking can benefit from GMC. Existing GMC algorithms rely on sequentially processing consecutive frames, by estimating the transformation mapping the two frames, and obtaining a composite transformation to a global motion compensated coordinate. Sequential GMC suffers from temporal drift of frames from the accurate global coordinate, due to either error accumulation or sporadic failures of motion estimation at a few frames. We propose a temporally robust global motion compensation (TRGMC) algorithm which performs accurate and stable GMC, despite complicated and long-term camera motion. TRGMC densely connects pairs of frames, by matching local keypoints of each frame. A joint alignment of these frames is formulated as a novel keypoint-based congealing problem, where the transformation of each frame is updated iteratively, such that the spatial coordinates for the start and end points of matched keypoints are identical. Experimental results demonstrate that TRGMC has superior performance in a wide range of scenarios.

TRGMC Flowchart

Flowchart of the TRGMC algorithm.

Sample Results

TRGMC Source Code

TRGMC implementation in Matlab may be downloaded from here.

This project was supported by the research gift from TechSmith Inc.

If you use TRGMC code, please cite to the ECCV 2016 paper:

Publications

  • Temporally Robust Global Motion Compensation by Keypoint-based Congealing
    Seyed Morteza Safdarnejad, Yousef Atoum, Xiaoming Liu
    Proc. European Conference on Computer Vision (ECCV 2016), Amsterdam, The Netherlands, Oct. 2016
    Bibtex | PDF | Code
  • @inproceedings{ temporally-robust-global-motion-compensation-by-keypoint-based-congealing,
      author = { Seyed Morteza Safdarnejad and Yousef Atoum and Xiaoming Liu },
      title = { Temporally Robust Global Motion Compensation by Keypoint-based Congealing },
      booktitle = { Proc. European Conference on Computer Vision },
      address = { Amsterdam, The Netherlands },
      month = { October },
      year = { 2016 },
    }