We hear much about reinforcement, which is, in my opinion a poor choice of a term to describe a type of artificial network that continues to acquire or improve its behavioral information in natura (during operations in the field). Reinforcement in learning theory is a term used to describe repetitious incentivization to increase the durability of learned material. In machine learning, the term has been twisted to denote the application of feedback in operations, a form of re-entrant back propagation.
Qualitatively, corrective signaling in field operations can supply information to a network to make only two types of functional adjustments.
- Adjustments to what is considered the optimum, beginning with the optimum found during training prior to deployment
- Testing of entirely new areas of the parameter space for hint of new optima that have formed, any of which might currently qualify or soon qualify as the global optimum.
(By optima and optimum, we mean minima and global minimum in the surface that describes the disparity between ideal system behavior and current system behavior. This surface is sometimes termed the error surface, applying an over-simplifying analogy from the mathematical discipline of curve fitting.)
The Importance of Doubt
The second of the two above could aptly be termed doubt.
Perhaps all neural nets should have one or more parallel doubting networks that can test remote areas of the search space for more promising optima. In a parallel computing environment, this might be a matter of provisioning and not significantly reduce the throughput of the primary network, yet provide a layer of reliability not found without the doubtful parallel networks.
What Shows More Intelligence?
Which is more important in actual field use of AI? The ability to reinforce what is already learned or the ability to create a minority opinion, doubt the status quo, and determine if it is not a more appropriate behavioral alternative than that which was reinforced.
A Helpful Pool of Water Analogy
During a short period of time, a point on the surface of the water may be the lowest point in a pool. With adjustments based on gradient (what is so inappropriately called reinforcement) the local well can be tracked so the low point can be maintained without any discrete jumps to other minima in the surface. However the local well may cease being the global minimum at some point in time, whereby a new search for a global minimum must ensue.
It may be that the new global minimum is across several features on the surface of the pool and cannot be found with gradient descent.
More interestingly, the appearance of new global minima can be tracked and reasonable projections can be made such that discrete and substantial jumps in parametric state can be accomplished without large jumps in disparity (where the system misbehaves badly for a period).
Circling Back to the Question
Which is more important, doubt or reinforcement?