3 votes

In logistic regression, why is the binary cross-entropy loss function convex?

The $L_{CE}$ that you provided is binary cross-entropy, the factor $y$ and $(1-y)$ is because $y$ is binary $({0,1})$, careful with the name next time. The cross-entropy loss should have form: $$L_{CE}...
user avatar
  • 677
3 votes

How to get more accuracy of the logistic regression model?

Try Rectification Improve the features available to your model, Remove some of the NOISE present in the data. In audio data, a common way to do this is to smooth the data and then rectify it so that ...
user avatar
  • 609
2 votes

How many parameter would there be in a logistic regression model used to classify reviews into "good" or "bad"?

Nope! Our number of coefficients will be driven by the vocabulary, and we'll use each of those 10K samples to estimate values for those coefficients - so, 'just' 100K samples. However, word ...
user avatar
2 votes

What is the definition of the hinge loss function?

The hinge loss/error function is the typical loss function used for binary classification (but it can also be extended to multi-class classification) in the context of support vector machines, ...
user avatar
  • 33k
2 votes
Accepted

Can you use machine learning for data with binary outcomes?

Of course, you can use AI (especially Deep Learning) in your application. Your covariates will be the input to your AI model and the model should predict the probability of presence. The model has no ...
user avatar
  • 430
2 votes
Accepted

Why doesn't the set $\{ -2, +2 \}$ in $E(X) = (y − \text{sign}\{\overline{W} \cdot \overline{X} \}) \in \{ −2, +2 \}$ include $0$?

It is important to note that the exact statement is the eqation given below can never be 0 for misclassified points in $ S^+$ $$ E(X) = (y - \text{sign}\{\overline{W} \cdot \overline{X}\}) $$ And $S+$ ...
user avatar
  • 71
2 votes
Accepted

Could I just choose the other (non-predicted) class when the accuracy is low?

The short answer is no, you shouldn't do that. There is a "distribution shift" thing when you have different x-y relation on the validation set then on the train set. The distribution shift ...
user avatar
2 votes
Accepted

What are pros and cons of using a multi-head neural network versus a single neural network for multi-label classification?

If I understood things correctly: You have a task which you need to estimate two values, gender and age. Your question revolves about the difference between networks which share layers for both inputs,...
user avatar
2 votes

If we want to classify something as either a cat/dog or neither, do we need 2 or 3 classes?

If you want to determine if something is either a cat/dog or neither you need 2 classes: one for dog or cat, and one for anything else. However, if you assign all cats and dogs to the same class $...
user avatar
  • 113
2 votes

Does summing up word vectors destroy their meaning?

Summing up a sequence of word vector maybe used in practice sometimes. However, the operation of addition is non-reversible, meaning that once you sum up a few numbers, you cannot get the original ...
user avatar
  • 1,715
1 vote

In logistic regression, why is the binary cross-entropy loss function convex?

If you find the Hessian matrix (the matrix of second order derivatives) for the binary cross entropy loss function, you'll see that it is positive semidefinite for any possible value of the parameters....
user avatar
1 vote

Prediction of continuous variable based on threshold

I don't think there's a hard rule here, just do what makes sense. Do a standard binary classification task. Assuming the label itself actually contains information (and it's not just there to ...
user avatar
  • 121
1 vote

Which pre-processing steps are necessary for Deep Learning models to solve a document classification problem?

It depends on the type of model you use and the task, you are attempting to solve. Almost all the preprocessing steps that you mention remove some information from the text. If you think the removed ...
user avatar
  • 141
1 vote

Given a dataset of people with and without cancer, should I split it into training and test datasets such that the same person is not in both?

we are recognizing the disease, not the person. If you're training a computer vision model with only images and no auxiliary information then a randomized sampling should be enough to prevent the ...
user avatar
1 vote
Accepted

How to arrange test dataset distribution for an imbalanced classification problem?

The test set should represents the "real" data distribution your model will tackle once deployed and used in real applications. So the quick answer is yes, the test data should be imbalanced,...
user avatar
1 vote

When doing binary classification with neural networks, how can I order the importance of the features for a class?

Two popular methods I’ve seen done: 1) For each feature, remove it and run the model and see the impact it has on the result. The idea is that the larger the impact, the more pertinent it was to ...
user avatar
  • 2,229
1 vote
Accepted

Which loss function should I use for binary classification?

There are several loss functions that you can use for binary classification. For example, you could use the binary cross-entropy or the hinge loss functions. See, for example, the tutorials Binary ...
user avatar
  • 33k
1 vote
Accepted

Is it appropriate to use a softmax activation with a categorical crossentropy loss?

Let's first recap the definition of the binary cross-entropy (BCE) and the categorical cross-entropy (CCE). Here's the BCE (equation 4.90 from this book) $$-\sum_{n=1}^{N}\left( t_{n} \ln y_{n}+\left(...
user avatar
  • 33k
1 vote

Which approach should I use to classify points above and below a sine function $y(x) = A + B \sin(Cx)$?

You can try using Fourier basis functions to transform your observable variables and then apply a general linear regression model. To clarify, if you have pairs of observables $(y_i, x_i)$ where $y_i$ ...
user avatar
  • 2,216
1 vote

Why are CNN binary classifier output probability distributions often skewed?

Yes, due to this issue, you should use temperature scaling after training your model. It will calibrate your probability and you will start to get the same kind of distributions. Here are a good ...
user avatar
1 vote

Support Vector Machine Convert optimisation problem from argmax to argmin

So actually I managed to get hold of my lecturer to explain the argmax to argmin conversion. Generally speaking maximising $\frac{1}{||w||}$ is identical to minimising $||w||$. As $||w||$ in $\frac{1}{...
1 vote

Does summing up word vectors destroy their meaning?

But because the inputs have to have a fixed length Do they? Why? The go-to strategy would be to use an RNN (possibly with LSTM or GRUs, but probably not necessary) and train it to process input ...
user avatar
1 vote
Accepted

How can I use Generative Adversarial Networks to solve the imbalanced class problem?

In my experience, GANs work really well for the scenario of semi-supervised learning, where you don't necessarily have labels for all your class $B$ data, but you do have a balanced dataset. In my (...
user avatar
  • 176
1 vote

Why is my fine-tuned YOLO model detecting other objects as a human?

So you have a network pretrained on 80 classes. I also assume that one of these classes are human (or else this is just not the way to go*) I suspect that the final layer contains 80 labels, correct? ...
user avatar
1 vote

If we want to classify something as either a cat/dog or neither, do we need 2 or 3 classes?

As far as generalization error is concerned, you are better off by learning the data distribution of (A and B) classes using unsupervised criterion. If you capture the underlying factors that ...
user avatar
1 vote
Accepted

If we want to classify something as either a cat/dog or neither, do we need 2 or 3 classes?

The best approach may be to have a cat, dog, and neither class (3 classes total) and go with a regression approach — specifically, outputting the probabilities of each class for any given input. From ...
user avatar

Only top scored, non community-wiki answers of a minimum length are eligible