Reading the high-level descriptions of backpropagation and predictive coding, they don't sound so drastically different. What is the key difference between these techniques?
I am currently reading the following paper if that helps ground the explanation:
Predictive Coding-based Deep Dynamic Neural Network for Visuomotor Learning
I would say that these concepts are quite different, even though they might have a few things in common (or might be vaguely related).
Back-propagation is an algorithm used (in machine learning) to compute the gradient of a function with respect to its parameters. This gradient is then used by algorithms, like gradient descent, to update the parameters of the model (e.g. a neural network).
Roughly, predictive coding is a general (neuroscience) theory of how the brain builds an internal model of the external world, of how it continuously predicts the sensory inputs (from the world) using its current model of the external world, and of how it updates this internal model once the sensory inputs are actually received.
You could think of the output of an ML model, while it is being trained (using gradient descent with back-propagation), as a prediction associated with the inputs. However, note that the output of the model is often not a prediction of the actual input, but e.g. a label (which is, nonetheless, associated with the input). Furthermore, it is the model that produces an output and not the back-propagation algorithm (even though the back-propagation algorithm is often used to train such models, e.g. neural networks). We could think of back-propagation as the way of updating these predictions associated with the inputs, but, anyway, it is well known (and accepted) that our brain does not perform back-propagation, but we learn in an associative fashion (Hebbian learning).
