Neural network with logical hidden layer - how to train it? Is it policy gradient problem? Chaining NNs?

Question

I am doing neural machine translation task from language S to language T via interlingua L. So - there is the structure:

S ->
encoding of S (crisp) ->
S-L encoder -> S-L decoder ->
encoding of L (non-crisp, coming from decoder) ->
L ->
encoding of L (crisp) ->
L-T encoder -> L-T decoder ->
encoding of T (non-crisp, coming from decoder) ->
T

All of this can be implemented in Pytorch more or less adapting the usual encoder-decoder NMT. So, the layer of interlingua L acts as a somehow symbolic/discrete layer inside the whole S-L-T neural network. My question is - how such system can be trained in end-to-end (S-T) manner? The gradient propagates from the T to the L and at the L one should do some kind of symbolic gradient? I. e. one should be able do compute the difference L1-L2?

I am somehow confused by such setting. My question is - is there similar networks which contain the symbolic representation as the intermediate layer and how one can train such system. I have heard about policy gradients but are they relevant to my setting?

Essentially - if I denote some neural network by symbols x(Wi)y, then the training of this network means, that I change Wi and x stays intact. I.e. the last member of backpropagation equation has the form d.../dw1. But if I combine (chain!) 2 neural networks x(Wi)y-y(Wj)z, then the the last backpropagation term for the y(Wj)z has the form (d.../dw1+d.../dy) and hence both the w1 and y should be changed/updated by the gradient descent too. So, doesn't some ambiguity arise here? Is such chaining of neural networks possible? Is is possible to train end-to-end chains of neural networks?

I am also thinking about use of evolutionary training.

Answer 1

The chain rule applies here as usual, and the term symbolic gradient is interesting.

How such might apply will depend much on the nature of the policy layer representation and the connections of that layer or layers to other non-policy layers.

• A binary threshold layer may feed a logical inference layer.
• A sigmoid function might feed a fuzzy logic layer.
• The logical layer could be a production system adapted to look like a layer.
• The logical layer might be more like a PLD (programmable logic device).
• The output of the logical layer might be mapped one to one with a perceptron or an LSTM layer

In the example in the question, we can't, from the information given in the question, assume that discrete means a discrete approximation of a curve, since the term used is discrete/symbolic. Since the nature of logic cannot be generalized in a differentiable closed form, it may be that it has to be probabilistically represented in closed form so a derivative may be obtained, if that is possible in the specific case under study.

Back propagation is a feedback mechanism to provide corrective signaling to the portions of the system that must be corrected. The hope is that the direction of the correction during back propagation leads to convergence on the global minimum of loss without severe delay, which is not always easy to make happen, thus Gaussian noise injection explicitly or indirectly through mini-batching, momentum, multiple initialization states, parallel searches with different hyper-parameters, and other techniques.

One technique used quite a bit in analog systems and entering use in digital systems in recent years is the idea of multiple corrective mechanisms. Control theory for instrumentation and countermeasure aeronautics is thick with this concept. For the case given in this question, it can look like this, with $f$ representing feed forward layer with some activation function, $\ell$ representing some logical inference container, $p$ is the parameter set for the function of the same subscript, $\epsilon$ is the evaluation function (error, loss, value, benefit), $b$ representing the corrective feedback using the back propagation, the Jacobian, and the chain rule.

$$ f_1 \quad \rightarrow \;\quad \ell \quad \rightarrow \quad\ f_2 \quad \rightarrow \quad\ \epsilon \\ p_1 \leftarrow \, b \leftarrow \ell\, ; \quad\quad\quad \; p_2 \, \leftarrow b \leftarrow \epsilon \\ \quad\quad\quad\quad\quad \ell \quad\quad \leftarrow \quad\quad\quad\quad\quad \epsilon $$

In the last line $\epsilon$ is simply a fact (piece of information) passed into the a rules engine session or a floating point representation of a feedback signal assigned as a probability to a rule in a fuzzy logical container. In the first half of the second line, the logical unit is responsible for providing the evaluation function output.

In Norbert Wiener's Cybernetics, he states the following¹ with regard to automating the steering of ships via rudder control, from which cybernetics got its name.

In the important book of MacColl², we have an example of a complicated system which can be stabilized by two feedbacks but not by one.

He then procures the differential equations and proves MacColl's conclusion. That conclusion is used in nearly every instrumentation amplifier circuit on a chip sold today, if not all of them. There is no way to get the desired stability without the multiple feedback paths.

Similarly, it may not be possible to stabilize the convergence of $f_1, \ell, and f_2$ via a single mechanism of back-propagation using $\epsilon$ and the Jacobians to descend a gradient.

---

References

[1] Norbert Wiener's *Cybernetics, 1948, MIT, p 106 in the 1996 printing

[2] LeRoy A. MacColl. Fundamental Theory of Servomechanisms. Van Nostrand, New York, 1945