This article on human activity recognition states that 1D convolutional neural networks work the best on the classification of human activities using data from the accelerometer. But I think that human activities, like swinging the arm, are sequential actions and they require LSTMs.

So, which one should be more effective for this task: CNNs or LSTMs? In other words, is spatial learning required or sequence learning?


1 Answer 1


Human activity recognition, for example as done with the Sports-1M Dataset, is a classification task. Given a video, say which activity / action was executed.

So it is a special case about video classification.

human activities like swinging the arm are sequential actions and they require LSTMs

This is wrong. As a simple example, think of handwriting recognition as done on write-math.com: I receive a sequence of points (x, y) -- the pen-tip -- and classify which symbol is written. So it is a sequence, but still there a plenty of ways to use a simple multi-layer perceptron (MLP) for it. Two big groups of approaches:

  • Feature-Engineering: Instead of giving an arbitrary number of coordinates, one can normalize it to a fixed amount. This is what is currently done on write-math.com and works pretty well (see my Bachelors thesis)
  • Space Transformation: Similar to the feature engineering, but more extreme. Instead of looking at (variants of) coordinates, one can look at the image which is there in the end. This way one can apply CNNs.

You can do similar things for movies:

  • Multiple Classifications: Classify based on single frames and average the results
  • Feature-Engineering: Normalize the length to a fixed number; interpret the fixed-length video as an image with more channels.
  • Use CNNs as feature extractors / for dimensionality reduction and LSTM for the sequence classification.

You might be interested in Beyond Short Snippets: Deep Networks for Video Classification

Efficiency vs Effectiveness

Model A is more effective than model B, if its optimizarion criterion is better.

Model A is more efficient than model B, if it uses less resources (e.g. memory) to achieve the same optimization value (e.g. same accuracy / MSE)


You must log in to answer this question.