Large displacement 3D scene flow with occlusion reasoning

Andrei Zanfir, Cristian Sminchisescu

Research output: Chapter in Book/Report/Conference proceedingPaper in conference proceedingpeer-review

Abstract

The emergence of modern, affordable and accurate RGB-D sensors increases the need for single view approaches to estimate 3-dimensional motion, also known as scene flow. In this paper we propose a coarse-to-fine, dense, correspondence-based scene flow formulation that relies on explicit geometric reasoning to account for the effects of large displacements and to model occlusion. Our methodology enforces local motion rigidity at the level of the 3d point cloud without explicitly smoothing the parameters of adjacent neighborhoods. By integrating all geometric and photometric components in a single, consistent, occlusion-aware energy model, defined over overlapping, image-adaptive neighborhoods, our method can process fast motions and large occlusions areas, as present in challenging datasets like the MPI Sintel Flow Dataset, recently augmented with depth information. By explicitly modeling large displacements and occlusion, we can handle difficult sequences which cannot be currently processed by state of the art scene flow methods. We also show that by integrating depth information into the model, we can obtain correspondence fields with improved spatial support and sharper boundaries compared to the state of the art, large-displacement optical flow methods.

Original languageEnglish
Title of host publicationProceedings - 2015 IEEE International Conference on Computer Vision, ICCV 2015
PublisherIEEE - Institute of Electrical and Electronics Engineers Inc.
Pages4417-4425
Number of pages9
Volume11-18-December-2015
ISBN (Electronic)9781467383912
DOIs
Publication statusPublished - 2016 Feb 17
Event15th IEEE International Conference on Computer Vision, ICCV 2015 - Santiago, Chile
Duration: 2015 Dec 112015 Dec 18

Conference

Conference15th IEEE International Conference on Computer Vision, ICCV 2015
Country/TerritoryChile
CitySantiago
Period2015/12/112015/12/18

Subject classification (UKÄ)

  • Computer Vision and Robotics (Autonomous Systems)

Fingerprint

Dive into the research topics of 'Large displacement 3D scene flow with occlusion reasoning'. Together they form a unique fingerprint.

Cite this