0
$\begingroup$

Suppose I have a simple linear layer $y = xA^T + b$ that is part of a neural network trained on some dataset. The weight matrix $A$ for this layer has the shape [num_outputs, num_inputs].

For each layer input, I would like to find a value between 0 and 1, based on the weight matrix, representing the significance of that input to the layer output.

Intuitively, if the values in the i-th column of the weight matrix are close to 0, then the significance of i-th inputh should also be close to 0. Conversely, if the values are close to maximum or minimum of the entire weight matrix, the significance should approach 1.

This statistic should also adequately recognize cases where the vast majority of values in a column are close to 0, but at least one is not. Then the significance of such an input should not be close to 0, because it is important for a single neuron that detects, for example, an edge case.

Can anyone point me in the right direction?

$\endgroup$
2
  • $\begingroup$ Conversely, if the values are close to maximum or minimum of the entire weight matrix, the significance should approach 1. How minimum values can be assigned weightage of 1? $\endgroup$
    – hanugm
    Jul 30 at 22:59
  • $\begingroup$ @hanugm Weights can also be negative numbers. An input with a weight of -0.2 has much less impact on the output than an input with a weight of -2.4. $\endgroup$
    – Kasia
    Jul 31 at 5:35

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.