On geometric features for skeleton-based action recognition using multilayer LSTM networks

Songyang Zhang, Xiaoming Liu, Jun Xiao

    Research output: ResearchConference contribution

    • 3 Citations

    Abstract

    RNN-based approaches have achieved outstanding performance on action recognition with skeleton inputs. Currently these methods limit their inputs to coordinates of joints and improve the accuracy mainly by extending RNN models to spatial domains in various ways. While such models explore relations between different parts directly from joint coordinates, we provide a simple universal spatial modeling method perpendicular to the RNN model enhancement. Specifically, we select a set of simple geometric features, motivated by the evolution of previous work. With experiments on a 3-layer LSTM framework, we observe that the geometric relational features based on distances between joints and selected lines outperform other features and achieve state-of-Art results on four datasets. Further, we show the sparsity of input gate weights in the first LSTM layer trained by geometric features and demonstrate that utilizing joint-line distances as input require less data for training.

    LanguageEnglish (US)
    Title of host publicationProceedings - 2017 IEEE Winter Conference on Applications of Computer Vision, WACV 2017
    PublisherInstitute of Electrical and Electronics Engineers Inc.
    Pages148-157
    Number of pages10
    ISBN (Electronic)9781509048229
    DOIs
    StatePublished - May 11 2017
    Event17th IEEE Winter Conference on Applications of Computer Vision, WACV 2017 - Santa Rosa, United States
    Duration: Mar 24 2017Mar 31 2017

    Other

    Other17th IEEE Winter Conference on Applications of Computer Vision, WACV 2017
    CountryUnited States
    CitySanta Rosa
    Period3/24/173/31/17

    Profile

    Multilayers
    Experiments

    ASJC Scopus subject areas

    • Computer Science Applications
    • Computer Vision and Pattern Recognition

    Cite this

    Zhang, S., Liu, X., & Xiao, J. (2017). On geometric features for skeleton-based action recognition using multilayer LSTM networks. In Proceedings - 2017 IEEE Winter Conference on Applications of Computer Vision, WACV 2017 (pp. 148-157). [7926607] Institute of Electrical and Electronics Engineers Inc.. DOI: 10.1109/WACV.2017.24

    On geometric features for skeleton-based action recognition using multilayer LSTM networks. / Zhang, Songyang; Liu, Xiaoming; Xiao, Jun.

    Proceedings - 2017 IEEE Winter Conference on Applications of Computer Vision, WACV 2017. Institute of Electrical and Electronics Engineers Inc., 2017. p. 148-157 7926607.

    Research output: ResearchConference contribution

    Zhang, S, Liu, X & Xiao, J 2017, On geometric features for skeleton-based action recognition using multilayer LSTM networks. in Proceedings - 2017 IEEE Winter Conference on Applications of Computer Vision, WACV 2017., 7926607, Institute of Electrical and Electronics Engineers Inc., pp. 148-157, 17th IEEE Winter Conference on Applications of Computer Vision, WACV 2017, Santa Rosa, United States, 3/24/17. DOI: 10.1109/WACV.2017.24
    Zhang S, Liu X, Xiao J. On geometric features for skeleton-based action recognition using multilayer LSTM networks. In Proceedings - 2017 IEEE Winter Conference on Applications of Computer Vision, WACV 2017. Institute of Electrical and Electronics Engineers Inc.2017. p. 148-157. 7926607. Available from, DOI: 10.1109/WACV.2017.24
    Zhang, Songyang ; Liu, Xiaoming ; Xiao, Jun. / On geometric features for skeleton-based action recognition using multilayer LSTM networks. Proceedings - 2017 IEEE Winter Conference on Applications of Computer Vision, WACV 2017. Institute of Electrical and Electronics Engineers Inc., 2017. pp. 148-157
    @inbook{6ca497b5efdf46bca02b65e9adedeacc,
    title = "On geometric features for skeleton-based action recognition using multilayer LSTM networks",
    abstract = "RNN-based approaches have achieved outstanding performance on action recognition with skeleton inputs. Currently these methods limit their inputs to coordinates of joints and improve the accuracy mainly by extending RNN models to spatial domains in various ways. While such models explore relations between different parts directly from joint coordinates, we provide a simple universal spatial modeling method perpendicular to the RNN model enhancement. Specifically, we select a set of simple geometric features, motivated by the evolution of previous work. With experiments on a 3-layer LSTM framework, we observe that the geometric relational features based on distances between joints and selected lines outperform other features and achieve state-of-Art results on four datasets. Further, we show the sparsity of input gate weights in the first LSTM layer trained by geometric features and demonstrate that utilizing joint-line distances as input require less data for training.",
    author = "Songyang Zhang and Xiaoming Liu and Jun Xiao",
    year = "2017",
    month = "5",
    doi = "10.1109/WACV.2017.24",
    pages = "148--157",
    booktitle = "Proceedings - 2017 IEEE Winter Conference on Applications of Computer Vision, WACV 2017",
    publisher = "Institute of Electrical and Electronics Engineers Inc.",

    }

    TY - CHAP

    T1 - On geometric features for skeleton-based action recognition using multilayer LSTM networks

    AU - Zhang,Songyang

    AU - Liu,Xiaoming

    AU - Xiao,Jun

    PY - 2017/5/11

    Y1 - 2017/5/11

    N2 - RNN-based approaches have achieved outstanding performance on action recognition with skeleton inputs. Currently these methods limit their inputs to coordinates of joints and improve the accuracy mainly by extending RNN models to spatial domains in various ways. While such models explore relations between different parts directly from joint coordinates, we provide a simple universal spatial modeling method perpendicular to the RNN model enhancement. Specifically, we select a set of simple geometric features, motivated by the evolution of previous work. With experiments on a 3-layer LSTM framework, we observe that the geometric relational features based on distances between joints and selected lines outperform other features and achieve state-of-Art results on four datasets. Further, we show the sparsity of input gate weights in the first LSTM layer trained by geometric features and demonstrate that utilizing joint-line distances as input require less data for training.

    AB - RNN-based approaches have achieved outstanding performance on action recognition with skeleton inputs. Currently these methods limit their inputs to coordinates of joints and improve the accuracy mainly by extending RNN models to spatial domains in various ways. While such models explore relations between different parts directly from joint coordinates, we provide a simple universal spatial modeling method perpendicular to the RNN model enhancement. Specifically, we select a set of simple geometric features, motivated by the evolution of previous work. With experiments on a 3-layer LSTM framework, we observe that the geometric relational features based on distances between joints and selected lines outperform other features and achieve state-of-Art results on four datasets. Further, we show the sparsity of input gate weights in the first LSTM layer trained by geometric features and demonstrate that utilizing joint-line distances as input require less data for training.

    UR - http://www.scopus.com/inward/record.url?scp=85020190655&partnerID=8YFLogxK

    UR - http://www.scopus.com/inward/citedby.url?scp=85020190655&partnerID=8YFLogxK

    U2 - 10.1109/WACV.2017.24

    DO - 10.1109/WACV.2017.24

    M3 - Conference contribution

    SP - 148

    EP - 157

    BT - Proceedings - 2017 IEEE Winter Conference on Applications of Computer Vision, WACV 2017

    PB - Institute of Electrical and Electronics Engineers Inc.

    ER -