Suppose I have a simple linear layer $y = xA^T + b$ that is part of a neural network trained on some dataset. The weight matrix $A$ for this layer has the shape
For each layer input, I would like to find a value between 0 and 1, based on the weight matrix, representing the significance of that input to the layer output.
Intuitively, if the values in the i-th column of the weight matrix are close to 0, then the significance of i-th inputh should also be close to 0. Conversely, if the values are close to maximum or minimum of the entire weight matrix, the significance should approach 1.
This statistic should also adequately recognize cases where the vast majority of values in a column are close to 0, but at least one is not. Then the significance of such an input should not be close to 0, because it is important for a single neuron that detects, for example, an edge case.
Can anyone point me in the right direction?