Training a neural network to output the conditional probability of an event when the training data output is only binary

Question

I have a dataset with hundreds of thousands of training examples. There are 27 input variables and one output variable which is always a 0 or a 1, based on whether an event happened or not.

My network therefore has 27 inputs and 1 output. I want the network's output to be a confidence guess of how likely the event is to happen, for example if the output is 0.23 then that represents that the network thinks the event has a 23% chance of happening.

I am using back propagation to train the neural network. It does appear to work well and the network outputs a higher number when the event is more likely and a lower number when the event is less likely.

Would it be a valid concern that my training data only has 0 or 1 values as outputs, when this is not truly what I want the network to output?

My concern comes from the fact that back propagation attempts to reduce the square of the error between the network's output, and the value of the output in the training data, which is always a 0 or a 1. Because it is the square of the error it is trying to reduce, I'm concerned that it's probability output may not be a linear mapping to the true probability of the event happening based on the 27 inputs it is seeing.

Is this a valid concern? And are there any techniques I can use to get a neural network to output a linear confidence guess between 0 and 1 when my test data only has outputs of 0 or 1?

I am using the sigmoid activation function for all of my neurons, would there be a better choice of activation function for this problem?

Edit: Thanks to Xpector's answer, I now understand that not all back propagation aims to reduce the square of the error, it depends on the loss function used. I am including a part of the back propagation code I have used here which calculates the error:

var neuronOutput = layerOutputs[i];
var error = (neuronOutput - desiredOutput[i]);
errors[i] = error * Maths.SigmoidDerivative(neuronOutput);

This is from an open source RProp implementation. I am not sure what loss function is being used here.

Answer 1

To sort the terms a bit, back propagation (repeatedly applying the chain rule of differentiation) is an implementation of stochastic gradient descent, which is an implementation of optimization. It is not necessarily the square of the error that is reduced by optimization - it is up to us to choose the loss function.

The loss function should ideally mirror the utility function (what you are truly want to optimize), but it needs a useful derivative. For example, the 0-1 loss hasn't one.

I think that what is called linear confidence guess in the question is the conditional probability of the event, given the input. Then the binary cross entropy would be a valid choice of a loss function.

From deeplearningbook.org

The negative log-likelihood allows the model to estimate the conditional probability of the classes, given the input, and if the model can do that well, then it can pick the classes that yield the least classification error in expectation.