I have an idea for an RNN which has no separate internal memory state only an output. But there is a gate in which tells the neural network whether the output will be acted out in the physical world or it will be an internal thought. (It would also store its last, say, 10 outputs, so that it can have a memory of some kind.)
I think this would be quite realistic because human's either talk or think in an internal monologue, but don't do both. (It is hard to think and do things at the same time).
But I wonder how this gate will be activated. For example, when talking to someone familiar, this gate will be open, as you just say what's in your head. But for quiet contemplation time, this gate will be closed. And for thoughtful conversation, it will be open 50% of the time. So, I wonder if this gate would be controlled by the NN itself or be controlled from the environment?
I think there would be social pressure involved when talking to someone to keep the gate open. And likewise when in a library or a quiet place to keep the gate closed.
I wonder if there are some models like this out there already?