Fusion of Keypoint Tracking and Facial Landmark Detection for Real-Time Head Pose Estimation

Jilliam María Díaz Barros, Bruno Mirbach, Frederic Garcia, Kiran Varanasi, Didier Stricker

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

23 Citations (Scopus)

Abstract

In this paper, we address the problem of extreme head pose estimation from intensity images, in a monocular setup. We introduce a novel fusion pipeline to integrate into a dedicated Kalman Filter the pose estimated from a tracking scheme in the prediction stage and the pose estimated from a detection scheme in the correction stage. To that end, the measurement covariance of the Kalman Filter is updated in every frame. The tracking scheme is performed using a set of keypoints extracted in the area of the head along with a simple 3D geometric model. The detection scheme, on the other hand, relies on the alignment of facial landmarks in each frame combined with 3D features extracted on a head mesh. The head pose in each scheme is estimated by minimizing the reprojection error from the 3D-2D correspondences. By combining both frameworks, we extend the applicability of head pose estimation from facial landmarks to cases where these features are no longer visible. We compared the proposed method to other related approaches, showing that it can achieve state-of-the-art performance. We also demonstrate that our approach is suitable for cases with extreme head rotations and (self-) occlusions, besides being suitable for real time applications.

Original languageEnglish
Title of host publicationProceedings - 2018 IEEE Winter Conference on Applications of Computer Vision, WACV 2018
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages2028-2037
Number of pages10
ISBN (Electronic)9781538648865
DOIs
Publication statusPublished - 3 May 2018
Externally publishedYes
Event18th IEEE Winter Conference on Applications of Computer Vision, WACV 2018 - Lake Tahoe, United States
Duration: 12 Mar 201815 Mar 2018

Publication series

NameProceedings - 2018 IEEE Winter Conference on Applications of Computer Vision, WACV 2018
Volume2018-January

Conference

Conference18th IEEE Winter Conference on Applications of Computer Vision, WACV 2018
Country/TerritoryUnited States
CityLake Tahoe
Period12/03/1815/03/18

Fingerprint

Dive into the research topics of 'Fusion of Keypoint Tracking and Facial Landmark Detection for Real-Time Head Pose Estimation'. Together they form a unique fingerprint.

Cite this