I'm currently deeply invested in the Transformer Circuits thread in parallel with 3blue1brown's videos (chapter 7 on the MLP layer was released a day or two ago) to gain a better theoretical understanding of transformers, in particular relating to mechanistic interpretability. As I don't necessarily have a full CS/ML background, I sometimes don't really know how and where to ask for clarifications.
One issue I'm wondering about whether the residual stream can be "read" (in the sense of unembedded) at various steps (before/after self-attention, before/after MLP, in-between layers) and whether that would produce meaningful or at least interesting tokens. My intuition or current understanding is that at first, unembedding the token's residual stream would produce the token itself, successive steps of attention and MLP further and further nudging the unembedding from a (nearly) 1-hot token encoding to the softmax next-token prediction.
Is this intuition at least partially correct?
More broadly, since these authors often consider toy models, is there something like GPT-2-mini with published weights available online where I could manually "investigate" the embeddings in the residual stream?
