The paper Dynamic Routing Between Capsules uses the algorithm called "Dynamic Routing Between Capsules" to determine the coupling coefficients between capsules.

Why it can't be done by backpropagation?


Very interesting paper.

We can see that they have effectively modeled probabilistically the fuzzy existence of an object with various geometric attributes as objects in a scene with a hook to the region of pixels that correspond to its possible existence. I agree with the authors that this is a much more robust approach than CNN and LSTM and may compete well with the emerging attention based approaches.

Back propagation does seem to be involved, although that is not the focus of the paper.

Notice, "there is top-down feedback," and, "an appropriate parent in the layer above," which are hints to a hierarchical approach, and we can see this approach is not absent of its context in the overall feeding forward of input signalling to output, all of which must be trained as a whole. But just as with much older control systems, the end-to-end convergence is facilitated by (not replaced by) more local convergence of subsystems with their own control objectives.

The overview is shown in section 4 on page 4.

The entire thing could conceivably be done without capsules at all, as with the aforementioned designs, but this group is apparently showing improved results when stopping short of full convergence on the MNIST. No comparison is shown with LSTM, but even if LSTM was showing greater results, this direction of research is an excellent one because of the probabilitic way that it approaches an object.

Consider the classic case of waving back at someone who, as it turns out, was waving at someone behind you. The existence of an object or an action is necessarily probabilitic, and to construct the depth of network required to model all that complexity with ReLU is probably unrealistic as the expectations increase in AI system requirements.

| improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.