1
$\begingroup$

XOR data, without labels:

[[0,0],[0,1],[1,0],[1,1]]

I'm using this network for auto-classifying XOR data:

H1  <-- Dense(units=2, activation=relu)    #any activation here
Z   <-- Dense(units=2, activation=softmax) #softmax for 2 classes of XOR result
Out <-- Dense(units=2, activation=sigmoid) #sigmoid to return 2 values in (0,1)

There's a logical problem in the network, that is, Z represents 2 classes, however, the 2 classes can't be decoded back to 4 samples of XOR data.

How to fix the network above to auto-classify XOR data, in unsupervised manner?

Improve this question
$\endgroup$
  • $\begingroup$ Programming issues are off-topic here. You've been a user on this site for a long time, so I recommend you read our on-topic page: ai.stackexchange.com/help/on-topic. $\endgroup$ – nbro♦ Jul 22 '20 at 11:18
  • $\begingroup$ nana, i meant to ask about theory, ie. how to design an autoencoder to group up unsupervised data $\endgroup$ – datdinhquoc Jul 23 '20 at 4:09
  • $\begingroup$ Ok. Please, next time make sure to clarify that, especially, when you add code to your post. $\endgroup$ – nbro♦ Jul 25 '20 at 13:57
1
$\begingroup$

How to fix the network above to auto-classify XOR data, in unsupervised manner?

This cannot be done, except accidentally.

Unsupervised learning cannot replace or emulate supervised learning.

As a thought experiment, consider why you would expect the network to discover XOR, when simply considering outputs rounded to binary, you could equally find AND, OR, NAND, NOR or any of the 16 possible mapping functions from input to output. All of the possible maps are equally valid functions, and there is no reason why a discovered function mapping should become any one of them by preference.

Unsupervised learning approaches typically find patterns that optimise some measure across the dataset without using labelled data. Clustering is a classic example, and auto-encoding is sometimes considered unsupervised because there is no separate label (although the term self-supervised is also used, because there is still technically a label used in training, it happens to equal the input).

You cannot use auto-encoding approaches here anyway, because XOR needs to map $\{0,1\} \times \{0,1\} \rightarrow \{0,1\}$

You could potentially use a loss function based on how close to a 0 or 1 any output is. That should cause the network to converge to one of the 16 possible binary functions, based on random initialisation. For example, you could use $y(1-y)$ as the loss.

Improve this answer
$\endgroup$
  • $\begingroup$ >Unsupervised learning cannot replace or emulate supervised learning. Actually, I disagree with it a bit. If you find a pattern/clusters with USL, and then label the pattern/clusters with some label, then you emulate SL a bit. $\endgroup$ – Daniel Wiczew Jul 25 '20 at 21:03
  • $\begingroup$ @DanielWiczew I think that is strectching definitions, but I suppose it is a matter of taste. I don't think I will include in the answer though, because it won't solve OP's problem. $\endgroup$ – Neil Slater Jul 25 '20 at 21:09
  • $\begingroup$ @NeilSlater tks, i now use different fixed init weights for different grouping discovery :) $\endgroup$ – datdinhquoc Jul 26 '20 at 3:14

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.