Here's a simple image classifier implemented in TensorFlow Keras (right click to open in new tab): https://colab.research.google.com/github/tensorflow/docs/blob/master/site/en/tutorials/quickstart/advanced.ipynb
I altered it a bit to fit with my 2-class dataset. And the output layer is:
Dense(2, activation=tf.nn.softmax);
The loss function and optimiser are still the same as in the example in the link above.
loss_fn = tf.losses.SparseCategoricalCrossentropy();
optimizer = tf.optimizers.Adam();
I wish to turn it into a classifier with single output neuron as I have only 2 classes in dataset, and sigmoid does the 2 classes good. Tried some combinations of output activation functions + loss functions + optimisers, but the network doesn't work any more (ie. it doesn't converge).
For example, this doesn't work:
//output layer
Dense(1, activation=tf.sigmoid);
//loss and optim
loss_fn = tf.losses.mse;
optimizer = tf.optimizers.Adagrad(1e-1);
Which combination of output activation + loss + optimiser should work for the single-output-neuron model? And generically, which loss functions and optimisers should pair well?