On geometric features for skeleton-based action recognition using multilayer LSTM networks

Songyang Zhang, Xiaoming Liu, Jun Xiao

Research output: Chapter in Book/Report/Conference proceedingConference contribution

  • 16 Citations

Abstract

RNN-based approaches have achieved outstanding performance on action recognition with skeleton inputs. Currently these methods limit their inputs to coordinates of joints and improve the accuracy mainly by extending RNN models to spatial domains in various ways. While such models explore relations between different parts directly from joint coordinates, we provide a simple universal spatial modeling method perpendicular to the RNN model enhancement. Specifically, we select a set of simple geometric features, motivated by the evolution of previous work. With experiments on a 3-layer LSTM framework, we observe that the geometric relational features based on distances between joints and selected lines outperform other features and achieve state-of-Art results on four datasets. Further, we show the sparsity of input gate weights in the first LSTM layer trained by geometric features and demonstrate that utilizing joint-line distances as input require less data for training.

LanguageEnglish (US)
Title of host publicationProceedings - 2017 IEEE Winter Conference on Applications of Computer Vision, WACV 2017
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages148-157
Number of pages10
ISBN (Electronic)9781509048229
DOIs
StatePublished - May 11 2017
Event17th IEEE Winter Conference on Applications of Computer Vision, WACV 2017 - Santa Rosa, United States
Duration: Mar 24 2017Mar 31 2017

Other

Other17th IEEE Winter Conference on Applications of Computer Vision, WACV 2017
CountryUnited States
CitySanta Rosa
Period3/24/173/31/17

Profile

Multilayers
Experiments

ASJC Scopus subject areas

  • Computer Science Applications
  • Computer Vision and Pattern Recognition

Cite this

Zhang, S., Liu, X., & Xiao, J. (2017). On geometric features for skeleton-based action recognition using multilayer LSTM networks. In Proceedings - 2017 IEEE Winter Conference on Applications of Computer Vision, WACV 2017 (pp. 148-157). [7926607] Institute of Electrical and Electronics Engineers Inc.. DOI: 10.1109/WACV.2017.24

On geometric features for skeleton-based action recognition using multilayer LSTM networks. / Zhang, Songyang; Liu, Xiaoming; Xiao, Jun.

Proceedings - 2017 IEEE Winter Conference on Applications of Computer Vision, WACV 2017. Institute of Electrical and Electronics Engineers Inc., 2017. p. 148-157 7926607.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Zhang, S, Liu, X & Xiao, J 2017, On geometric features for skeleton-based action recognition using multilayer LSTM networks. in Proceedings - 2017 IEEE Winter Conference on Applications of Computer Vision, WACV 2017., 7926607, Institute of Electrical and Electronics Engineers Inc., pp. 148-157, 17th IEEE Winter Conference on Applications of Computer Vision, WACV 2017, Santa Rosa, United States, 3/24/17. DOI: 10.1109/WACV.2017.24
Zhang S, Liu X, Xiao J. On geometric features for skeleton-based action recognition using multilayer LSTM networks. In Proceedings - 2017 IEEE Winter Conference on Applications of Computer Vision, WACV 2017. Institute of Electrical and Electronics Engineers Inc.2017. p. 148-157. 7926607. Available from, DOI: 10.1109/WACV.2017.24
Zhang, Songyang ; Liu, Xiaoming ; Xiao, Jun. / On geometric features for skeleton-based action recognition using multilayer LSTM networks. Proceedings - 2017 IEEE Winter Conference on Applications of Computer Vision, WACV 2017. Institute of Electrical and Electronics Engineers Inc., 2017. pp. 148-157
@inproceedings{6ca497b5efdf46bca02b65e9adedeacc,
title = "On geometric features for skeleton-based action recognition using multilayer LSTM networks",
abstract = "RNN-based approaches have achieved outstanding performance on action recognition with skeleton inputs. Currently these methods limit their inputs to coordinates of joints and improve the accuracy mainly by extending RNN models to spatial domains in various ways. While such models explore relations between different parts directly from joint coordinates, we provide a simple universal spatial modeling method perpendicular to the RNN model enhancement. Specifically, we select a set of simple geometric features, motivated by the evolution of previous work. With experiments on a 3-layer LSTM framework, we observe that the geometric relational features based on distances between joints and selected lines outperform other features and achieve state-of-Art results on four datasets. Further, we show the sparsity of input gate weights in the first LSTM layer trained by geometric features and demonstrate that utilizing joint-line distances as input require less data for training.",
author = "Songyang Zhang and Xiaoming Liu and Jun Xiao",
year = "2017",
month = "5",
day = "11",
doi = "10.1109/WACV.2017.24",
language = "English (US)",
pages = "148--157",
booktitle = "Proceedings - 2017 IEEE Winter Conference on Applications of Computer Vision, WACV 2017",
publisher = "Institute of Electrical and Electronics Engineers Inc.",

}

TY - GEN

T1 - On geometric features for skeleton-based action recognition using multilayer LSTM networks

AU - Zhang,Songyang

AU - Liu,Xiaoming

AU - Xiao,Jun

PY - 2017/5/11

Y1 - 2017/5/11

N2 - RNN-based approaches have achieved outstanding performance on action recognition with skeleton inputs. Currently these methods limit their inputs to coordinates of joints and improve the accuracy mainly by extending RNN models to spatial domains in various ways. While such models explore relations between different parts directly from joint coordinates, we provide a simple universal spatial modeling method perpendicular to the RNN model enhancement. Specifically, we select a set of simple geometric features, motivated by the evolution of previous work. With experiments on a 3-layer LSTM framework, we observe that the geometric relational features based on distances between joints and selected lines outperform other features and achieve state-of-Art results on four datasets. Further, we show the sparsity of input gate weights in the first LSTM layer trained by geometric features and demonstrate that utilizing joint-line distances as input require less data for training.

AB - RNN-based approaches have achieved outstanding performance on action recognition with skeleton inputs. Currently these methods limit their inputs to coordinates of joints and improve the accuracy mainly by extending RNN models to spatial domains in various ways. While such models explore relations between different parts directly from joint coordinates, we provide a simple universal spatial modeling method perpendicular to the RNN model enhancement. Specifically, we select a set of simple geometric features, motivated by the evolution of previous work. With experiments on a 3-layer LSTM framework, we observe that the geometric relational features based on distances between joints and selected lines outperform other features and achieve state-of-Art results on four datasets. Further, we show the sparsity of input gate weights in the first LSTM layer trained by geometric features and demonstrate that utilizing joint-line distances as input require less data for training.

UR - http://www.scopus.com/inward/record.url?scp=85020190655&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85020190655&partnerID=8YFLogxK

U2 - 10.1109/WACV.2017.24

DO - 10.1109/WACV.2017.24

M3 - Conference contribution

SP - 148

EP - 157

BT - Proceedings - 2017 IEEE Winter Conference on Applications of Computer Vision, WACV 2017

PB - Institute of Electrical and Electronics Engineers Inc.

ER -