Using True Positive as a Cost Function

I wanted to use True Positive (and True Negative) in my cost function to make to modify the ROC shape of my classifier. Someone told me and I read that it is not differentiable and therefore not usable as a cost function for a neural network.

In the example where 1 is positive and 0 negative I deduce the following equation for True Positive ($$\hat y = prediction, y = label$$):

$$TP = \bf(\hat{y}^Ty)$$ $$\frac{\partial TP}{\partial \bf y} = \bf y$$

The following for True Negative: $$TN = \bf(\hat{y}-1)^T(y-1)$$ $$\frac{\partial TN}{\partial \bf y} = \bf \hat y^T -1$$

The False Positive: $$FP = - \bf (\hat y^T-1) y$$ $$\frac{\partial FP}{\partial \bf y} = - \bf ( \hat y^T - 1)$$

The False Negative: $$FN = \bf \hat y^T (y-1)$$ $$\frac{\partial FN}{\partial \bf y} = \bf \hat y$$

All equations seem differentiable to me. Can someone explain where I went wrong?

• As an aside, it is more usual to have $\hat{y}$ as the prediction and $y$ as the true value. – Neil Slater Aug 29 at 7:31

The vector functions for true positive, false positive etc all make use of the "magic" numbers $$0$$ and $$1$$ used to represent Boolean values. They are convenience methods that you can use in a numerical library, but you do need to be aware of the fundamental Boolean nature of the data. The $$0$$ and $$1$$ values allow the maths for calculating TP et al, but are not fundamental to it, they are a representation.
Your derivations of gradients for the functions you give seem correct, barring the odd typo. However, the gradient doesn't really apply to the value of $$\mathbf{y}$$, because all components of $$\mathbf{y}$$ are either $$0$$ or $$1$$. The idea that you could increase $$\mathbf{y}$$ slightly where $$\mathbf{\hat{y}}$$ is $$1$$ in order to increase the value of the TP metric slightly has no basis. Instead the only valid changes to make an improvement are to modify FN values from $$0$$ to $$1$$ exactly.
You could probably still use your derivations as a gradient for optimisation (it would not be the only time in machine learning that something does not quite apply theoretically but you could still use it in practice). However, you then immediately hit the problem of how the values of $$\mathbf{y}$$ have been discretised to $$0$$ or $$1$$ - that function will not be differentiable, and it will prevent you back propagating your gradients to the neural network weights that you want to change. If you fix that follow-on problem using a smoother function (e.g. a sigmoid) then you are likely to end up with something close to either cross-entropy loss or the perceptron update step.