3 votes

In logistic regression, why is the binary cross-entropy loss function convex?

The $L_{CE}$ that you provided is binary cross-entropy, the factor $y$ and $(1-y)$ is because $y$ is binary $({0,1})$, careful with the name next time. The cross-entropy loss should have form: $$L_{CE}...
  • 857
3 votes

How to get more accuracy of the logistic regression model?

Try Rectification Improve the features available to your model, Remove some of the NOISE present in the data. In audio data, a common way to do this is to smooth the data and then rectify it so that ...
  • 625
3 votes

What is the definition of the hinge loss function?

The hinge loss/error function is the typical loss function used for binary classification (but it can also be extended to multi-class classification) in the context of support vector machines, ...
  • 35k
2 votes

How many parameter would there be in a logistic regression model used to classify reviews into "good" or "bad"?

Nope! Our number of coefficients will be driven by the vocabulary, and we'll use each of those 10K samples to estimate values for those coefficients - so, 'just' 100K samples. However, word ...
2 votes
Accepted

Can you use machine learning for data with binary outcomes?

Of course, you can use AI (especially Deep Learning) in your application. Your covariates will be the input to your AI model and the model should predict the probability of presence. The model has no ...
  • 420
2 votes
Accepted

Why doesn't the set $\{ -2, +2 \}$ in $E(X) = (y − \text{sign}\{\overline{W} \cdot \overline{X} \}) \in \{ −2, +2 \}$ include $0$?

It is important to note that the exact statement is the eqation given below can never be 0 for misclassified points in $ S^+$ $$ E(X) = (y - \text{sign}\{\overline{W} \cdot \overline{X}\}) $$ And $S+$ ...
  • 61
2 votes
Accepted

Could I just choose the other (non-predicted) class when the accuracy is low?

The short answer is no, you shouldn't do that. There is a "distribution shift" thing when you have different x-y relation on the validation set then on the train set. The distribution shift ...
2 votes

Does summing up word vectors destroy their meaning?

Summing up a sequence of word vector maybe used in practice sometimes. However, the operation of addition is non-reversible, meaning that once you sum up a few numbers, you cannot get the original ...
  • 1,715
2 votes
Accepted

What are pros and cons of using a multi-head neural network versus a single neural network for multi-label classification?

If I understood things correctly: You have a task which you need to estimate two values, gender and age. Your question revolves about the difference between networks which share layers for both inputs,...
2 votes

If we want to classify something as either a cat/dog or neither, do we need 2 or 3 classes?

If you want to determine if something is either a cat/dog or neither you need 2 classes: one for dog or cat, and one for anything else. However, if you assign all cats and dogs to the same class $...
  • 138
1 vote

In logistic regression, why is the binary cross-entropy loss function convex?

If you find the Hessian matrix (the matrix of second order derivatives) for the binary cross entropy loss function, you'll see that it is positive semidefinite for any possible value of the parameters....
1 vote
Accepted

Why do smaller weights converge faster for RNNs?

There is no magic value that work for every network but in general: too large initial weights lead to exploding gradients (i.e. no convergence) too small initial weights lead to vanishing gradients (...
1 vote

ML algorithm suggestion for databases that change a lot with time after model training

It sounds to me that while the data is changing every week, it is still in the same domain. That should make things easier. You need a neural network that generalises well. Faster RCNN with ResNet as ...
1 vote

Prediction of continuous variable based on threshold

I don't think there's a hard rule here, just do what makes sense. Do a standard binary classification task. Assuming the label itself actually contains information (and it's not just there to ...
  • 121
1 vote

Which pre-processing steps are necessary for Deep Learning models to solve a document classification problem?

It depends on the type of model you use and the task, you are attempting to solve. Almost all the preprocessing steps that you mention remove some information from the text. If you think the removed ...
  • 151
1 vote

Given a dataset of people with and without cancer, should I split it into training and test datasets such that the same person is not in both?

we are recognizing the disease, not the person. If you're training a computer vision model with only images and no auxiliary information then a randomized sampling should be enough to prevent the ...
1 vote
Accepted

How to arrange test dataset distribution for an imbalanced classification problem?

The test set should represents the "real" data distribution your model will tackle once deployed and used in real applications. So the quick answer is yes, the test data should be imbalanced,...
1 vote

In logistic regression, why is the binary cross-entropy loss function convex?

I'm unable to comment on previous answers because I'm new to ai.stackexchange and don't have enough clout points. So I'm writing my comment as an answer instead. Unless I'm missing something, I ...
1 vote

Is binary classification using CNN possible if the training data only consists of one class?

While it won't work as you've possibly imagined it, you might find that implementing it as an autoencoder will allow you to train on one class and then identify things that are "not that." ...
1 vote

When doing binary classification with neural networks, how can I order the importance of the features for a class?

Two popular methods I’ve seen done: 1) For each feature, remove it and run the model and see the impact it has on the result. The idea is that the larger the impact, the more pertinent it was to ...
  • 2,299
1 vote
Accepted

Which loss function should I use for binary classification?

There are several loss functions that you can use for binary classification. For example, you could use the binary cross-entropy or the hinge loss functions. See, for example, the tutorials Binary ...
  • 35k
1 vote
Accepted

Is it appropriate to use a softmax activation with a categorical crossentropy loss?

Let's first recap the definition of the binary cross-entropy (BCE) and the categorical cross-entropy (CCE). Here's the BCE (equation 4.90 from this book) $$-\sum_{n=1}^{N}\left( t_{n} \ln y_{n}+\left(...
  • 35k
1 vote

Which approach should I use to classify points above and below a sine function $y(x) = A + B \sin(Cx)$?

You can try using Fourier basis functions to transform your observable variables and then apply a general linear regression model. To clarify, if you have pairs of observables $(y_i, x_i)$ where $y_i$ ...
  • 2,236
1 vote

Why are CNN binary classifier output probability distributions often skewed?

Yes, due to this issue, you should use temperature scaling after training your model. It will calibrate your probability and you will start to get the same kind of distributions. Here are a good ...
1 vote

Support Vector Machine Convert optimisation problem from argmax to argmin

So actually I managed to get hold of my lecturer to explain the argmax to argmin conversion. Generally speaking maximising $\frac{1}{||w||}$ is identical to minimising $||w||$. As $||w||$ in $\frac{1}{...
1 vote

Does summing up word vectors destroy their meaning?

But because the inputs have to have a fixed length Do they? Why? The go-to strategy would be to use an RNN (possibly with LSTM or GRUs, but probably not necessary) and train it to process input ...
1 vote
Accepted

How can I use Generative Adversarial Networks to solve the imbalanced class problem?

In my experience, GANs work really well for the scenario of semi-supervised learning, where you don't necessarily have labels for all your class $B$ data, but you do have a balanced dataset. In my (...
  • 176
1 vote

Why is my fine-tuned YOLO model detecting other objects as a human?

So you have a network pretrained on 80 classes. I also assume that one of these classes are human (or else this is just not the way to go*) I suspect that the final layer contains 80 labels, correct? ...
1 vote

If we want to classify something as either a cat/dog or neither, do we need 2 or 3 classes?

As far as generalization error is concerned, you are better off by learning the data distribution of (A and B) classes using unsupervised criterion. If you capture the underlying factors that ...
1 vote
Accepted

If we want to classify something as either a cat/dog or neither, do we need 2 or 3 classes?

The best approach may be to have a cat, dog, and neither class (3 classes total) and go with a regression approach — specifically, outputting the probabilities of each class for any given input. From ...

Only top scored, non community-wiki answers of a minimum length are eligible