The chain rule applies here as usual, and the term symbolic gradient is interesting.
How such might apply will depend much on the nature of the policy layer representation and the connections of that layer or layers to other non-policy layers.
- A binary threshold layer may feed a logical inference layer.
- A sigmoid function might feed a fuzzy logic layer.
- The logical layer could be a production system adapted to look like a layer.
- The logical layer might be more like a PLD (programmable logic device).
- The output of the logical layer might be mapped one to one with a perceptron or an LSTM layer
In the example in the question, we can't, from the information given in the question, assume that discrete means a discrete approximation of a curve, since the term used is discrete/symbolic. Since the nature of logic cannot be generalized in a differentiable closed form, it may be that it has to be probabilistically represented in closed form so a derivative may be obtained, if that is possible in the specific case under study.
Back propagation is a feedback mechanism to provide corrective signaling to the portions of the system that must be corrected. The hope is that the direction of the correction during back propagation leads to convergence on the global minimum of loss without severe delay, which is not always easy to make happen, thus Gaussian noise injection explicitly or indirectly through mini-batching, momentum, multiple initialization states, parallel searches with different hyper-parameters, and other techniques.
One technique used quite a bit in analog systems and entering use in digital systems in recent years is the idea of multiple corrective mechanisms. Control theory for instrumentation and countermeasure aeronautics is thick with this concept. For the case given in this question, it can look like this, with $f$ representing feed forward layer with some activation function, $\ell$ representing some logical inference container, $p$ is the parameter set for the function of the same subscript, $\epsilon$ is the evaluation function (error, loss, value, benefit), $b$ representing the corrective feedback using the back propagation, the Jacobian, and the chain rule.
$$ f_1 \quad \rightarrow \;\quad \ell \quad \rightarrow \quad\ f_2 \quad \rightarrow \quad\ \epsilon \\
p_1 \leftarrow \, b \leftarrow \ell\, ; \quad\quad\quad \; p_2 \, \leftarrow b \leftarrow \epsilon \\
\quad\quad\quad\quad\quad \ell \quad\quad \leftarrow \quad\quad\quad\quad\quad \epsilon
In the last line $\epsilon$ is simply a fact (piece of information) passed into the a rules engine session or a floating point representation of a feedback signal assigned as a probability to a rule in a fuzzy logical container. In the first half of the second line, the logical unit is responsible for providing the evaluation function output.
In Norbert Wiener's Cybernetics, he states the following1 with regard to automating the steering of ships via rudder control, from which cybernetics got its name.
In the important book of MacColl2, we have an example of a complicated system which can be stabilized by two feedbacks but not by one.
He then procures the differential equations and proves MacColl's conclusion. That conclusion is used in nearly every instrumentation amplifier circuit on a chip sold today, if not all of them. There is no way to get the desired stability without the multiple feedback paths.
Similarly, it may not be possible to stabilize the convergence of $f_1, \ell, and f_2$ via a single mechanism of back-propagation using $\epsilon$ and the Jacobians to descend a gradient.
 Norbert Wiener's *Cybernetics, 1948, MIT, p 106 in the 1996 printing
 LeRoy A. MacColl. Fundamental Theory of Servomechanisms. Van Nostrand, New York, 1945