We tackle a novel scenario of spatio-temporal alignment of seuqences referred to as Nonoverlapping Sequences (NOS). NOS are captured by multiple freely panning handheld cameras whose field of views (FOV) might have no direct spatial overlap. With the popularity of mobile sensors, NOS rise when multiple cooperative users capture a public event to create a panoramic video, or when consolidating multiple footages of an incident into a single video.
To tackle this novel scenario, we first spatially align the sequences by reconstructing the background of each sequence and registering these backgrounds, even if the backgrounds are not overlapping. Given the spatial alignment, we temporally synchronize the sequences, such that the trajectories of moving objects (e.g., cars or pedestrians) are consistent across sequences. Experimental results demonstrate the performance of our algorithm in this novel and challenging scenario, quantitatively and qualitatively.
(a) Top view of spatio-temporal FOV of two moving cameras; Non-overlapping sequences (NOS) may not even cover some common spatial region over the progression of time, i.e, no overall spatial overlap will exist. (b) Spatio-temporal alignment of NOS results in displaying sequences in a common coordinate and at the correct time shift.