18
votes
Accepted
Why has the cross-entropy become the classification standard loss function and not Kullback-Leibler divergence?
When it comes to a classification problem in machine learning, the cross-entropy and the KL divergence are equal.
As already stated in the question, the general formula is this:
$$H(p, q) = H(p) + D_{...
- 2,017
15
votes
Accepted
When is deep learning overkill?
It's all about Return On Investment. If DL is "worth doing", it's not overkill.
If the cost of using DL (computer cycles, storage, training time) is acceptable, and the data available to train it is ...
- 679
12
votes
When is deep learning overkill?
Deep learning is powerful but it is not a superior method than bayesian. They work well in what they are designed to do:
Use deep learning:
Cost for computation is much cheaper than cost of sampling ...
- 1,421
11
votes
Accepted
How should the neural network deal with unexpected inputs?
This is a very important problem that is usually overlooked. In fact, when training a neural network, there's often the implicit assumption that the data is independent and identically distributed, i....
- 41.4k
10
votes
Accepted
How does text classification reduce manpower costs?
There are several advantages:
Some text classification systems are much more accurate than 50%. For example, most spam classification systems are 99.9% accurate, or more. There will be little value ...
- 9,392
9
votes
What makes neural networks so good at predictions?
Neural networks are good at classifying. In some situations that comes down to prediction, but not necessarily.
The mathematical reason for the neural networks prowess at classifying is the ...
- 4,295
7
votes
Accepted
Which paper introduced the term "softmax"?
The paper that appears to have introduced the term "softmax" is Training Stochastic Model Recognition Algorithms as Networks can Lead to Maximum Mutual Information Estimation of Parameters (...
- 41.4k
7
votes
Accepted
Unclear points regarding ROC curve in machine learning
The "PERFECT CLASSIFIER" line is perfect because at the optimal threshold (somewhere near the top left corner, with TPR = 1 and FPR = 0), it makes no classification errors. The non-zero FPR ...
- 5,387
6
votes
Do I need classification or regression to predict the availability of a user given some features?
Yes. For instance, the popular softmax regression gives you probability distribution for each class.
Yes. Softmax is a regression over a set of discrete classes.
We can use regression for ...
- 1,421
6
votes
Can I train a neural network incrementally given new daily data?
Yes, this is possible. Continuously extending your training data is known as incremental learning.
You might also want to take a look at transfer learning, in which you reuse a trained model for a ...
- 176
5
votes
Accepted
How to refine K-means clustering on a data set?
The usual parameters to adjust in a k-means:
Number of clusters (recall many clusters can have same label).
Distance definition (euclidean is the most basic, Gauss is an
improvement)
Selection of ...
- 1,293
5
votes
Accepted
What is the difference between imitation learning and classification done by experts?
Imitation learning is supervised learning applied to the RL setting.
In any general RL algorithm (such as Q-learning), the learning is done on the basis of the reward function. However, consider a ...
- 136
5
votes
Accepted
Can machine learning algorithms be used to differentiate between small differences in details between images?
Attentive Recurrent Comparators (2017) by Pranav Shyam et al. is an interesting paper that helps to answer the question you're wondering, along with a blog post that helps to describe it in easier ...
- 527
5
votes
How do I improve accuracy and know when to stop training?
Is there anything else I could do to improve accuracy for both training and testing?
Yes, of course, there are a lot of methods if you want to try to improve your accuracy, some that I can mention:
...
- 2,819
5
votes
Accepted
What are causative and exploratory attacks in Adversarial Machine Learning?
When someone is able to do a causative attack it means there is a mechanism by which they are able to input data into the network. Maybe a website where people can input their images and it outputs a ...
- 555
5
votes
Accepted
Can a deep neural network be trained to classify an integer N1 as being divisible by another integer N2?
There is a recent development in research that was looking into effectiveness of neural networks on arithmetic. Interestingly, feed-forward neural networks (MLPs) with various activation functions as ...
- 414
5
votes
Accepted
Can ML/DL solve my classification problem?
A simple sanity-check on whether an image classifier can perform a task in theory is:
Can a human expert, using the same image plus a list of catgeories that they are familiar with, perform the same ...
- 33.3k
5
votes
Accepted
Is it possible that Precision and Recall increase together?
They can increase together if your new classifier is indeed way better than your older one in terms of almost every metric you can imagine including the two scores, together with the F1-score, or even ...
- 5,387
5
votes
Unclear points regarding ROC curve in machine learning
An ROC curve, strictly speaking, does not represent "a classifier", but a scoring rule that can be thresholded at different levels to yield many different classifiers. So, the line labeled &...
- 368
4
votes
Accepted
How many training example text classifier needs to be trained?
As a general rule of thumb I typically use 10*(# of features) for shallow machine learning models such as Naive Bayes with only 2 classes.
So it all depends on the number of features you will be ...
- 470
4
votes
How to determine if an Amazon review is likely to be fake using text classification
This will not be that hard of a problem once you have a lot of training data. But, before you have a lot of training data, you will need to get some training data one way or another. You will need a ...
- 873
4
votes
How to implement an "unknown" class in multi-class classification with neural networks?
The usual way to implement this would be to add the new class with data examples.
Some things you need to address:
Sourcing new data for your "other" class.
Ensuring the amount and variation of ...
- 33.3k
4
votes
Accepted
Why not use the MSE instead of the current logistic regression?
The mean squared error (MSE), $J(\theta) = \frac{1}{2m}\sum_{i=1}^m(h_\theta(x_i)-y_i)^2$, is not as appropriate as a cost function for classification, given that the MSE makes assumptions about the ...
- 1,106
4
votes
Accepted
Why is there more than one way of calculating the accuracy?
In machine learning, the accuracy is usually defined as the number of correct predictions divided by the total number of predictions. The correct predictions are the true positives ($\mathrm {TP}$) ...
- 41.4k
4
votes
How to go about classifying 1000 classes?
If you are asking for arbitrary ML task dealing with 1000+ classes the most straigtforward thing that comes to mind is the ImageNet - https://en.wikipedia.org/wiki/...
- 2,754
4
votes
Unclear points regarding ROC curve in machine learning
“Perfect classifier” might take it too far, but “perfect separation” is a good description, where “perfect separation” means that there is some threshold where all predictions corresponding to one ...
- 733
3
votes
Accepted
How to add more features to the input of a machine learning algorithm?
Data pre-processing and feature extraction are by far the most important part of any machine learning algorithm. It's even more important that the model you choose to do the classification.
...
- 470
3
votes
3
votes
What algorithm should I use to classify documents?
Text approach:
Use LDA (Latent Dirichlet Allocation). LDA is unsupervised. Feed it in corpuses of text from the various documents (i.e. OCR them and feed LDA the results of OCR). It will then cluster ...
- 86
Only top scored, non community-wiki answers of a minimum length are eligible