Pose-Invariant Face Alignment via CNN-Based Dense 3D Model Fitting

Amin Jourabloo, Xiaoming Liu

Research output: Contribution to journalArticle

  • 7 Citations

Abstract

Pose-invariant face alignment is a very challenging problem in computer vision, which is used as a prerequisite for many facial analysis tasks, e.g., face recognition, expression recognition, and 3D face reconstruction. Recently, there have been a few attempts to tackle this problem, but still more research is needed to achieve higher accuracy. In this paper, we propose a face alignment method that aligns an image with arbitrary poses, by combining the powerful cascaded CNN regressors, 3D Morphable Model (3DMM), and mirrorability constraint. The core of our proposed method is a novel 3DMM fitting algorithm, where the camera projection matrix parameters and 3D shape parameters are estimated by a cascade of CNN-based regressors. Furthermore, we impose the mirrorability constraint during the CNN learning by employing a novel loss function inside the siamese network. The dense 3D shape enables us to design pose-invariant appearance features for effective CNN learning. Extensive experiments are conducted on the challenging large-pose face databases (AFLW and AFW), with comparison to the state of the art.

LanguageEnglish (US)
Pages1-17
Number of pages17
JournalInternational Journal of Computer Vision
DOIs
StateAccepted/In press - Apr 19 2017

Profile

Face recognition
Computer vision
Cameras
Experiments

Keywords

  • Cascaded regressor
  • CNN
  • Dense model fitting
  • Mirrorability constraint
  • Pose-invariant face alignment

ASJC Scopus subject areas

  • Software
  • Computer Vision and Pattern Recognition
  • Artificial Intelligence

Cite this

Pose-Invariant Face Alignment via CNN-Based Dense 3D Model Fitting. / Jourabloo, Amin; Liu, Xiaoming.

In: International Journal of Computer Vision, 19.04.2017, p. 1-17.

Research output: Contribution to journalArticle

@article{b05ebccdfe764300ae3222c5e61d98e8,
title = "Pose-Invariant Face Alignment via CNN-Based Dense 3D Model Fitting",
abstract = "Pose-invariant face alignment is a very challenging problem in computer vision, which is used as a prerequisite for many facial analysis tasks, e.g., face recognition, expression recognition, and 3D face reconstruction. Recently, there have been a few attempts to tackle this problem, but still more research is needed to achieve higher accuracy. In this paper, we propose a face alignment method that aligns an image with arbitrary poses, by combining the powerful cascaded CNN regressors, 3D Morphable Model (3DMM), and mirrorability constraint. The core of our proposed method is a novel 3DMM fitting algorithm, where the camera projection matrix parameters and 3D shape parameters are estimated by a cascade of CNN-based regressors. Furthermore, we impose the mirrorability constraint during the CNN learning by employing a novel loss function inside the siamese network. The dense 3D shape enables us to design pose-invariant appearance features for effective CNN learning. Extensive experiments are conducted on the challenging large-pose face databases (AFLW and AFW), with comparison to the state of the art.",
keywords = "Cascaded regressor, CNN, Dense model fitting, Mirrorability constraint, Pose-invariant face alignment",
author = "Amin Jourabloo and Xiaoming Liu",
year = "2017",
month = "4",
day = "19",
doi = "10.1007/s11263-017-1012-z",
language = "English (US)",
pages = "1--17",
journal = "International Journal of Computer Vision",
issn = "0920-5691",
publisher = "Springer Netherlands",

}

TY - JOUR

T1 - Pose-Invariant Face Alignment via CNN-Based Dense 3D Model Fitting

AU - Jourabloo,Amin

AU - Liu,Xiaoming

PY - 2017/4/19

Y1 - 2017/4/19

N2 - Pose-invariant face alignment is a very challenging problem in computer vision, which is used as a prerequisite for many facial analysis tasks, e.g., face recognition, expression recognition, and 3D face reconstruction. Recently, there have been a few attempts to tackle this problem, but still more research is needed to achieve higher accuracy. In this paper, we propose a face alignment method that aligns an image with arbitrary poses, by combining the powerful cascaded CNN regressors, 3D Morphable Model (3DMM), and mirrorability constraint. The core of our proposed method is a novel 3DMM fitting algorithm, where the camera projection matrix parameters and 3D shape parameters are estimated by a cascade of CNN-based regressors. Furthermore, we impose the mirrorability constraint during the CNN learning by employing a novel loss function inside the siamese network. The dense 3D shape enables us to design pose-invariant appearance features for effective CNN learning. Extensive experiments are conducted on the challenging large-pose face databases (AFLW and AFW), with comparison to the state of the art.

AB - Pose-invariant face alignment is a very challenging problem in computer vision, which is used as a prerequisite for many facial analysis tasks, e.g., face recognition, expression recognition, and 3D face reconstruction. Recently, there have been a few attempts to tackle this problem, but still more research is needed to achieve higher accuracy. In this paper, we propose a face alignment method that aligns an image with arbitrary poses, by combining the powerful cascaded CNN regressors, 3D Morphable Model (3DMM), and mirrorability constraint. The core of our proposed method is a novel 3DMM fitting algorithm, where the camera projection matrix parameters and 3D shape parameters are estimated by a cascade of CNN-based regressors. Furthermore, we impose the mirrorability constraint during the CNN learning by employing a novel loss function inside the siamese network. The dense 3D shape enables us to design pose-invariant appearance features for effective CNN learning. Extensive experiments are conducted on the challenging large-pose face databases (AFLW and AFW), with comparison to the state of the art.

KW - Cascaded regressor

KW - CNN

KW - Dense model fitting

KW - Mirrorability constraint

KW - Pose-invariant face alignment

UR - http://www.scopus.com/inward/record.url?scp=85018467271&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85018467271&partnerID=8YFLogxK

U2 - 10.1007/s11263-017-1012-z

DO - 10.1007/s11263-017-1012-z

M3 - Article

SP - 1

EP - 17

JO - International Journal of Computer Vision

T2 - International Journal of Computer Vision

JF - International Journal of Computer Vision

SN - 0920-5691

ER -