How can I minimise the false positives?

Question

I have 50,000 samples. Of these 23,000 belong to the desired class $A$. I can sacrifice the number of instances that are classified as belonging to the desired class $A$. It will be enough for me to get 7000 instances in the desired class $A$, provided that most of these instances classified as belonging to $A$ really belong to the desired class $A$. How can I do this?

The following is the confusion matrix in the case the instances are perfectly classified.

[[23000   0]
 [  0 27000]]

But it is unlikely to obtain this confusion matrix, so I'm quite satisfied with the following confusion matrix.

[[7000   16000]
 [  500 26500]]

I am currently using the sklearn library. I mainly use algorithms based on decision trees, as they are quite fast in the calculation.

Answer 1

I think you're looking for the minimization of false positives, that is, the instances that are classified as belonging to the desired class (the positive part of false positives) but that do not actually belong to that class (the false part of false positives). In practice, given your constraints, you may want to maximize the precision, while maintaining a good recall.

In this answer to the question How can the model be tuned to improve precision, when precision is much more important than recall?, the user suggests performing a grid search (using sklearn.grid_search.GridSearchCV(clf, param_grid, scoring="precision")) to find the parameters of the model that maximize the precision. See also the question Classifier with adjustable precision vs recall.