1
$\begingroup$

We hear a lot today about how thought vectors are the Next Big Thing in AI, and how they serve as the underlying representation of thought/knowledge in ANN's. But how can one use thought vectors in other regimes, especially including symbolic logic / GOFAI? Could thought vectors be the "substrate" that binds together probabilistic approaches to AI and approaches that are rooted in logic?

$\endgroup$
2
$\begingroup$

I'll take a shot at answering this, though I'm no expert in Neural Nets or Deep Learning.

Given that practical thought vectors (TVs) don't yet exist, and may be impractical or impossible, I think answering your question will require a lot of conjecture and speculation. So here goes...

For thought vectors to be useful in or outside NNs, the vector values will have to be normalized, probably using the local context of the application problem's 'frame'. Without a NN to create new baseline vector values (weights) and to normalize them to match each new context, any non-NN mechanistic alternative means of employing TVs will somehow have to fill that void.

Could vector values created by NNs be used by an alternative technique? Could that technique also normalize them? Sure. We're just talking about turing-computable functions performed by NNs. If NNs aren't magic, then there should exist other means to compute the same results -- creating or editing, or employing TVs.

What might such an alternative to NNs be? Well, if its to shape vector weights, I suspect it too will have to learn those values through statistical iteration and feedback (as opposed to logical induction, say). I doubt such a mechanism exists yet, since it'd probably resemble NNs in sufficiently many ways that, thus far, it would have seemed too derivative of NNs to gain acceptance as sufficiently novel. Of course to be as powerful as deep nets, it too would have to propagate learning weights both forward and backward without incurring much error. Not an easy thing to accomplish.

Less ambitiously, could TVs be simply interpreted by another technique usefully? I think so. I can see several existing techniques, like decision trees or even expert systems, importing thought vectors and being shaped by them, and then function in accordance. But could these same techniques create or revise TVs usefully? Beyond a trivial extent, I'm doubtful. I think TVs are too complex a knowledge representation format for most general learning methods to both use and create/modify them, unless they employ an iterative statistical and feedback-based learning process, like those of NNs, which would allow novel and complex features to be learned and integrated into the vectors.

$\endgroup$
  • $\begingroup$ But could these same techniques create or revise TVs usefully? Beyond a trivial extent, I'm doubtful. I think TVs are too complex a knowledge representation format for most general learning methods to both use and create/modify them Yeah, I think that's the real question. If the answer is "yes" then thought vectors could really be the unifying layer where automated reasoning meets statistical pattern matching. Personally I hope the answer to that turns out to be "yes", but I'm not convinced one way or the other yet. $\endgroup$ – mindcrime Aug 31 '16 at 19:25

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.