Hierarchical on-line appearance-based tracking for 3d head pose, eyebrows, lips, eyelids and irises

Javier Orozco, Ognjen Rudovic, Jordi Gonzàlez, Maja Pantic

Research output: Contribution to journalArticleResearchpeer-review

35 Citations (Scopus)


In this paper, we propose an On-line Appearance-Based Tracker (OABT) for simultaneous tracking of 3D head pose, lips, eyebrows, eyelids and irises in monocular video sequences. In contrast to previously proposed tracking approaches, which deal with face and gaze tracking separately, our OABT can also be used for eyelid and iris tracking, as well as 3D head pose, lips and eyebrows facial actions tracking. Furthermore, our approach applies an on-line learning of changes in the appearance of the tracked target. Hence, the prior training of appearance models, which usually requires a large amount of labeled facial images, is avoided. Moreover, the proposed method is built upon a hierarchical combination of three OABTs, which are optimized using a Levenberg-Marquardt Algorithm (LMA) enhanced with line-search procedures. This, in turn, makes the proposed method robust to changes in lighting conditions, occlusions and translucent textures, as evidenced by our experiments. Finally, the proposed method achieves head and facial actions tracking in real-time. © 2013 Elsevier B.V.
Original languageEnglish
Pages (from-to)322-340
JournalImage and Vision Computing
Publication statusPublished - 1 Jan 2013


  • 3D face tracking
  • Eyelid tracking
  • Facial action tracking
  • Iris tracking
  • Levenberg-marquardt algorithm
  • Line-search optimization
  • On-line appearance models


Dive into the research topics of 'Hierarchical on-line appearance-based tracking for 3d head pose, eyebrows, lips, eyelids and irises'. Together they form a unique fingerprint.

Cite this