Question is in the title. Posts such as this and this mention how this would be possible. I have some colleagues who have anecdotally heard of this being done on very small transformer models but I have not been able to find any papers on this. I would be curious if anyone has resources on how well these models train and or perform.
