What are the strengths of the Hierarchical Temporal Memory model compared to competing models?

Question

What are the strengths of the Hierarchical Temporal Memory model compared to competing models such as 'traditional' Neural Networks as used in deep learning? And for those strengths are there other available models that aren't as bogged down by patents?

Answer 1

IMO, the greatest "strength" of HTM is that it is modeled after the human neocortex, which is the most intelligent thing we know of.

But to understand the importance of this simple idea one must contrast it with the most familiar form of AI - Neural Networks (NNs).

Traditional Neural Network AI has been under development for a long time and has many more people working on it than HTM. NNs are capable of performing a bewildering number of tasks, and the list of its accomplishments grows with every passing day.

However, NNs are not thinking. They perform their magic only after being trained on (typically) massive amounts of training data. Training a NN is essentially an advanced form of curve-fitting. If your training data encompasses closely enough what it encounters in new data then it will likely perform very well. However, if it encounters something new (which is sometimes difficult to know beforehand) then it can fail abysmally, and often in a way that humans would never fail.

One example I heard about was on a NN trained on millions of images that could briefly describe what was in new images it had never seen before. It performed fabulously - something like 95-97% accuracy. However, when it was shown an image of a baby holding a toothbrush, it said, "A boy holding a baseball bat." This is not a human-like error. Humans know the difference between a boy and a baby, and a bat and a toothbrush. This is just an example, but it reveals a fundamental problem of NNs - they are not thinking. Useful? Yes. Thinking? No.

Back to HTM. HTM is new and currently has only a handful of researchers working on it. It is "better" than NNs in only a small number of cases - it has a long way to go.

So if by "strengths" you're thinking about what tasks can currently be done better with HTM than with NNs, then most people should still chose NNs.

However, if by "strengths" you're thinking about what has the best chance of achieving general intelligence someday, then I would say hands-down it is HTM.

Answer 2

HTM is a credible theory about how the brain works, and how brain-like systems could be constructed in software. It includes:

• SDR (Sparse Distributed Representation), a means for representing just about any kind of sensory, intermediate or motor data, innately noise resistant and suited to recognising patterns
• TM (Temporal Memory), which can recognise SDRs in the context of other preceding SDRs, to learn new patterns "on the job" with no separate training phase
• SM (Sequence Memory), which can learn, remember and replay arbitrarily long sequences of SDRs.

ANNs are mature, commercially valuable pattern recognisers made possible by the confluence of vast amounts of computing power, data and commercial opportunity.

HTM is immature and just a fascinating toy, for now. But HTM might just put us on the path to the Holy Grail, true artificial general intelligence, and ANNs will never do that.