The browser you are using is not supported by this website. All versions of Internet Explorer are no longer supported, either by us or Microsoft (read more here: https://www.microsoft.com/en-us/microsoft-365/windows/end-of-ie-support).

Please use a modern browser to fully experience our website, such as the newest versions of Edge, Chrome, Firefox or Safari etc.

First and Second Order Dynamics in a Hierarchical SOM system for Action Recognition

På svenska

Author

Summary, in English

Human recognition of the actions of other humans is very efficient and is based on patterns of movements. Our theoretical starting point is that the dynamics of the joint movements is important to action categorization. On the basis of this theory, we present a novel action recognition system that employs a hierarchy of Self-Organizing Maps together with a custom supervised neural network that learns to categorize actions. The system preprocesses the input from a Kinect like 3D camera to exploit the information not only about joint positions, but also their first and second order dynamics. We evaluate our system in two experiments with publicly available datasets, and compare its performance to the performance with less sophisticated preprocessing of the input. The results show that including the dynamics of the actions improves the performance. We also apply an attention mechanism that focuses on the parts of the body that are the most involved in performing the actions.

Publishing year

2017-06-04

Language

English

Pages

574-585

Publication/Series

Applied Soft Computing

Volume

59

Document type

Journal article

Publisher

Elsevier

Topic

  • Computer Vision and Robotics (Autonomous Systems)

Status

Published

Project

  • What you say is what you did (WYSIWYD)
  • Ikaros: An infrastructure for system level modelling of the brain
  • Thinking in Time: Cognition, Communication and Learning

Research group

  • Lund University Cognitive Science (LUCS)

ISBN/ISSN/Other

  • ISSN: 1568-4946