0
$\begingroup$

The example is trying to predict wether coffe is well roasted or badly. 1 is good roasted and 0 is bad. The architecture is:architecture

Now I try to visualize the model. visualization Unit 1 has higher values when the duration of roasting is too little. Unit 2 has higher values for bad combinations of temperature and time. The blue-hatched regions show the layer outputted higher activations for the respective unit.

I used a threshold of 0.5 So I can conclude higher activations from layer 1 give an output class of 0. But as I used the sigmoid activation function I thought that large values give an output class of 1. And very small values give a 0, due to the S-shape.

$\endgroup$
3
  • $\begingroup$ Can you please highlight the question? Now sure what to answer. $\endgroup$
    – Chinmay
    Jul 14, 2023 at 17:27
  • $\begingroup$ Maybe you have negative weights in your output layer. What are the 3 weights that compute a2 from a1 ? $\endgroup$
    – Lelouch
    Jul 14, 2023 at 18:37
  • $\begingroup$ if it's a 2 layer network or more, you cannot reason anymore like this $\endgroup$
    – Alberto
    Jul 14, 2023 at 19:56

1 Answer 1

0
$\begingroup$

The sigmoid activation is defined as follows: $\sigma(x)=\frac{1}{1+e^{-x}}$, this means that for $x\to\infty$ the activation saturates to one whereas for $x\to-\infty$ it saturates to zero.

In your example you're trying to visualize the activation per neuron, in feature (or input) space. What I see is that if you sum the shaded blue regions (the ones with activation close to 1) you are able to identify the positive class quite well. Basically, each neuron is learning to threshold one side of the triangle where the negative class resides.

The output layer can sum these activation and rescale such that a further sigmoid, returns values close to $1$ for points that fall outside the triangle.

I can conclude higher activations from layer 1 give an output class of 0.

I don't see how you can say so. You shouldn't reason in feature space, but rather also account for the value of the weight that multiply each feature. Maybe, you want to say that, for a given neuron, a lower value of the feature corresponds to a higher activation but that depends on the sign of the weight the feature is multiplied with.

$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .