Unified multi-lateral filter for real-time depth map enhancement

Frederic Garcia*, Djamila Aouada, Bruno Mirbach, Thomas Solignac, Björn Ottersten

*Corresponding author for this work

Research output: Contribution to journalArticleResearchpeer-review

19 Citations (Scopus)

Abstract

This paper proposes a unified multi-lateral filter to efficiently increase the spatial resolution of low-resolution and noisy depth maps in real-time. Time-of-Flight (ToF) cameras have become a very promising alternative to stereo-based range sensing systems as they provide depth measurements at a high frame rate. However, there are actually two main drawbacks that restrict their use in a wide range of applications; namely, their fairly low spatial resolution as well as the amount of noise within the depth estimation. In order to address these drawbacks, we propose a new approach based on sensor fusion. That is, we couple a ToF camera of low-resolution with a 2-D camera of higher resolution to which the low-resolution depth map will be efficiently upsampled. In this paper, we first review the existing depth map enhancement approaches based on sensor fusion and discuss their limitations. We then propose a unified multi-lateral filter that accounts for the inaccuracy of depth edges position due to the low-resolution ToF depth maps. By doing so, unwanted artefacts such as texture copying and edge blurring are almost entirely eliminated. Moreover, the proposed filter is configurable to behave as most of the alternative depth enhancement approaches. Using a convolution-based formulation and data quantization and downsampling, the described filter has been effectively and efficiently implemented for dynamic scenes in real-time applications. The experimental results show a sensitive qualitative as well as quantitative improvement on raw depth maps, outperforming state-of-the-art multi-lateral filters.

Original languageEnglish
Pages (from-to)26-41
Number of pages16
JournalImage and Vision Computing
Volume41
DOIs
Publication statusPublished - 20 Jul 2015
Externally publishedYes

Keywords

  • Active sensing
  • Adaptive filters
  • Data fusion
  • Depth enhancement
  • Multi-modal sensors
  • Sensor fusion
  • Time-of-Flight

Fingerprint

Dive into the research topics of 'Unified multi-lateral filter for real-time depth map enhancement'. Together they form a unique fingerprint.

Cite this