# Symbolic “math” using trained networks

Does anyone work out ways of relating trained networks by symbolic logic?

For example: If I train a network on pictures of dogs, and I train a network on pictures of shirts. You could imagine that the simplest way of (without going through the process from scratch), identifying "dog AND shirt" would be to perform an AND operation on the last output of the individual cat & dog neural nets.

So "dog AND shirt" would amount to AND'ing the output of two nets (Which I believe is described here).

But this operation AND could be replaced with a more complicated operation. And in principle I could "train" a network to act as one of these operations.

For instance maybe I could figure out the net that describes some changeable output "X" being "on the shirt." This would be sort of like a "functional" in mathematics (in which we are operating are considering the behavior of a network whos input could be any network).

If I can figure out this "functional" then I would be able to use is symbolically and determine queries like "dog on the shirt"? - "cat on the shirt"?

It seems like to me there's a lot of sense to turn specific neural networks into more "abstract" objects - and that there would be a lot of power in doing so.

I think you would be interested in Neural-Symbolic Learning and Reasoning, a recent survey on the intersection between connectionist models (e.g. neural nets) and symbolic reasoning. It's a long paper, but it has a lot in it that is relevant to your question, if you're willing to dig. For example, it talks about learning networks that implement boolean logic.

This network searches for satisfying solutions for the weighted conjunctive normal form (CNF):$$\text{(¬X ∨ ¬Y ∨ Z) ∧ (X ∨ Y )}$$.

The closest thing I could find to the "functionals" you mentioned was a discussion on learning "relations" with neural networks. For this, it cited Neural-Symbolic Cognitive Reasoning, a book also about the intersection between neural networks and symbolic reasoning. In Chapter 10, it gives the example of teaching a neural network to learn the relation of "grandparent."

Our goal is to learn a description of grandparent from examples of the father, mother, and grandparent relations such as father(charles, william), mother(elisabeth, charles), and grandparent(elisabeth, william).

I could imagine something like this being done with the example inputs being image features from a CNN. Perhaps then you could start learning functionals like "on the shirt."