It is largely unknown how LLMs work inside. Has anyone scientifically tried to corrupt (open source) model's weights in an organized manner to maybe detect which parts of the model are doing what or are more or less important?

I mean something like setting the weights to zeros/random numbers in random (% of all weights) or organized (whole areas/layers etc.) manner and benchmarking the LLM to see how the results change.

Related reddit post.

  • 1
    $\begingroup$ isn't that page already answering your question? $\endgroup$
    – Alberto
    Commented Dec 3, 2023 at 21:08


You must log in to answer this question.