Pose-Invariant Face Alignment via CNN-Based Dense 3D Model Fitting

Amin Jourabloo, Xiaoming Liu

    Research output: Research - peer-reviewArticle

    Abstract

    Pose-invariant face alignment is a very challenging problem in computer vision, which is used as a prerequisite for many facial analysis tasks, e.g., face recognition, expression recognition, and 3D face reconstruction. Recently, there have been a few attempts to tackle this problem, but still more research is needed to achieve higher accuracy. In this paper, we propose a face alignment method that aligns an image with arbitrary poses, by combining the powerful cascaded CNN regressors, 3D Morphable Model (3DMM), and mirrorability constraint. The core of our proposed method is a novel 3DMM fitting algorithm, where the camera projection matrix parameters and 3D shape parameters are estimated by a cascade of CNN-based regressors. Furthermore, we impose the mirrorability constraint during the CNN learning by employing a novel loss function inside the siamese network. The dense 3D shape enables us to design pose-invariant appearance features for effective CNN learning. Extensive experiments are conducted on the challenging large-pose face databases (AFLW and AFW), with comparison to the state of the art.

    LanguageEnglish (US)
    Pages1-17
    Number of pages17
    JournalInternational Journal of Computer Vision
    DOIs
    StateAccepted/In press - Apr 19 2017

    Profile

    Face recognition
    Computer vision
    Cameras
    Experiments

    Keywords

    • Cascaded regressor
    • CNN
    • Dense model fitting
    • Mirrorability constraint
    • Pose-invariant face alignment

    ASJC Scopus subject areas

    • Software
    • Computer Vision and Pattern Recognition
    • Artificial Intelligence

    Cite this

    Pose-Invariant Face Alignment via CNN-Based Dense 3D Model Fitting. / Jourabloo, Amin; Liu, Xiaoming.

    In: International Journal of Computer Vision, 19.04.2017, p. 1-17.

    Research output: Research - peer-reviewArticle

    @article{b05ebccdfe764300ae3222c5e61d98e8,
    title = "Pose-Invariant Face Alignment via CNN-Based Dense 3D Model Fitting",
    abstract = "Pose-invariant face alignment is a very challenging problem in computer vision, which is used as a prerequisite for many facial analysis tasks, e.g., face recognition, expression recognition, and 3D face reconstruction. Recently, there have been a few attempts to tackle this problem, but still more research is needed to achieve higher accuracy. In this paper, we propose a face alignment method that aligns an image with arbitrary poses, by combining the powerful cascaded CNN regressors, 3D Morphable Model (3DMM), and mirrorability constraint. The core of our proposed method is a novel 3DMM fitting algorithm, where the camera projection matrix parameters and 3D shape parameters are estimated by a cascade of CNN-based regressors. Furthermore, we impose the mirrorability constraint during the CNN learning by employing a novel loss function inside the siamese network. The dense 3D shape enables us to design pose-invariant appearance features for effective CNN learning. Extensive experiments are conducted on the challenging large-pose face databases (AFLW and AFW), with comparison to the state of the art.",
    keywords = "Cascaded regressor, CNN, Dense model fitting, Mirrorability constraint, Pose-invariant face alignment",
    author = "Amin Jourabloo and Xiaoming Liu",
    year = "2017",
    month = "4",
    doi = "10.1007/s11263-017-1012-z",
    pages = "1--17",
    journal = "International Journal of Computer Vision",
    issn = "0920-5691",
    publisher = "Springer Netherlands",

    }

    TY - JOUR

    T1 - Pose-Invariant Face Alignment via CNN-Based Dense 3D Model Fitting

    AU - Jourabloo,Amin

    AU - Liu,Xiaoming

    PY - 2017/4/19

    Y1 - 2017/4/19

    N2 - Pose-invariant face alignment is a very challenging problem in computer vision, which is used as a prerequisite for many facial analysis tasks, e.g., face recognition, expression recognition, and 3D face reconstruction. Recently, there have been a few attempts to tackle this problem, but still more research is needed to achieve higher accuracy. In this paper, we propose a face alignment method that aligns an image with arbitrary poses, by combining the powerful cascaded CNN regressors, 3D Morphable Model (3DMM), and mirrorability constraint. The core of our proposed method is a novel 3DMM fitting algorithm, where the camera projection matrix parameters and 3D shape parameters are estimated by a cascade of CNN-based regressors. Furthermore, we impose the mirrorability constraint during the CNN learning by employing a novel loss function inside the siamese network. The dense 3D shape enables us to design pose-invariant appearance features for effective CNN learning. Extensive experiments are conducted on the challenging large-pose face databases (AFLW and AFW), with comparison to the state of the art.

    AB - Pose-invariant face alignment is a very challenging problem in computer vision, which is used as a prerequisite for many facial analysis tasks, e.g., face recognition, expression recognition, and 3D face reconstruction. Recently, there have been a few attempts to tackle this problem, but still more research is needed to achieve higher accuracy. In this paper, we propose a face alignment method that aligns an image with arbitrary poses, by combining the powerful cascaded CNN regressors, 3D Morphable Model (3DMM), and mirrorability constraint. The core of our proposed method is a novel 3DMM fitting algorithm, where the camera projection matrix parameters and 3D shape parameters are estimated by a cascade of CNN-based regressors. Furthermore, we impose the mirrorability constraint during the CNN learning by employing a novel loss function inside the siamese network. The dense 3D shape enables us to design pose-invariant appearance features for effective CNN learning. Extensive experiments are conducted on the challenging large-pose face databases (AFLW and AFW), with comparison to the state of the art.

    KW - Cascaded regressor

    KW - CNN

    KW - Dense model fitting

    KW - Mirrorability constraint

    KW - Pose-invariant face alignment

    UR - http://www.scopus.com/inward/record.url?scp=85018467271&partnerID=8YFLogxK

    UR - http://www.scopus.com/inward/citedby.url?scp=85018467271&partnerID=8YFLogxK

    U2 - 10.1007/s11263-017-1012-z

    DO - 10.1007/s11263-017-1012-z

    M3 - Article

    SP - 1

    EP - 17

    JO - International Journal of Computer Vision

    T2 - International Journal of Computer Vision

    JF - International Journal of Computer Vision

    SN - 0920-5691

    ER -