15
votes
Accepted
Why has the cross-entropy become the classification standard loss function and not Kullback-Leibler divergence?
When it comes to a classification problem in machine learning, the cross-entropy and the KL divergence are equal.
As already stated in the question, the general formula is this:
$$H(p, q) = H(p) + D_{...
- 1,877
14
votes
Accepted
When is deep learning overkill?
It's all about Return On Investment. If DL is "worth doing", it's not overkill.
If the cost of using DL (computer cycles, storage, training time) is acceptable, and the data available to train it is ...
- 671
12
votes
When is deep learning overkill?
Deep learning is powerful but it is not a superior method than bayesian. They work well in what they are designed to do:
Use deep learning:
Cost for computation is much cheaper than cost of sampling ...
- 1,390
11
votes
Accepted
How should the neural network deal with unexpected inputs?
This is a very important problem that is usually overlooked. In fact, when training a neural network, there's often the implicit assumption that the data is independent and identically distributed, i....
- 33.8k
10
votes
Accepted
How does text classification reduce manpower costs?
There are several advantages:
Some text classification systems are much more accurate than 50%. For example, most spam classification systems are 99.9% accurate, or more. There will be little value ...
- 8,877
8
votes
What makes neural networks so good at predictions?
Neural networks are good at classifying. In some situations that comes down to prediction, but not necessarily.
The mathematical reason for the neural networks prowess at classifying is the ...
- 4,007
6
votes
Do I need classification or regression to predict the availability of a user given some features?
Yes. For instance, the popular softmax regression gives you probability distribution for each class.
Yes. Softmax is a regression over a set of discrete classes.
We can use regression for ...
- 1,390
6
votes
Accepted
Can ML/DL solve my classification problem?
A simple sanity-check on whether an image classifier can perform a task in theory is:
Can a human expert, using the same image plus a list of catgeories that they are familiar with, perform the same ...
- 24k
5
votes
Accepted
How to refine K-means clustering on a data set?
The usual parameters to adjust in a k-means:
Number of clusters (recall many clusters can have same label).
Distance definition (euclidean is the most basic, Gauss is an
improvement)
Selection of ...
- 1,251
5
votes
How do I improve accuracy and know when to stop training?
Is there anything else I could do to improve accuracy for both training and testing?
Yes, of course, there are a lot of methods if you want to try to improve your accuracy, some that I can mention:
...
- 2,551
5
votes
Accepted
Can a deep neural network be trained to classify an integer N1 as being divisible by another integer N2?
There is a recent development in research that was looking into effectiveness of neural networks on arithmetic. Interestingly, feed-forward neural networks (MLPs) with various activation functions as ...
- 384
4
votes
Accepted
How does Pinterest decipher what's on unmarked pictures and categorize them?
One of the Pinterest's white paper about Human Curation and Convnets powering item-to-item recommendationsarxiv describes implementation of convolutional neural network (CNN) based visual features (...
- 10k
4
votes
Accepted
How many training example text classifier needs to be trained?
As a general rule of thumb I typically use 10*(# of features) for shallow machine learning models such as Naive Bayes with only 2 classes.
So it all depends on the number of features you will be ...
- 450
4
votes
How to determine if an Amazon review is likely to be fake using text classification
This will not be that hard of a problem once you have a lot of training data. But, before you have a lot of training data, you will need to get some training data one way or another. You will need a ...
- 833
4
votes
How to implement an "unknown" class in multi-class classification with neural networks?
The usual way to implement this would be to add the new class with data examples.
Some things you need to address:
Sourcing new data for your "other" class.
Ensuring the amount and variation of ...
- 24k
4
votes
Accepted
What is the difference between imitation learning and classification done by experts?
Imitation learning is supervised learning applied to the RL setting.
In any general RL algorithm (such as Q-learning), the learning is done on the basis of the reward function. However, consider a ...
- 106
4
votes
Accepted
Can machine learning algorithms be used to differentiate between small differences in details between images?
Attentive Recurrent Comparators (2017) by Pranav Shyam et al. is an interesting paper that helps to answer the question you're wondering, along with a blog post that helps to describe it in easier ...
- 463
4
votes
Can I train a neural network incrementally given new daily data?
Yes, this is possible. Continuously extending your training data is known as incremental learning.
You might also want to take a look at transfer learning, in which you reuse a trained model for a ...
- 156
4
votes
Accepted
Why not use the MSE instead of the current logistic regression?
The mean squared error (MSE), $J(\theta) = \frac{1}{2m}\sum_{i=1}^m(h_\theta(x_i)-y_i)^2$, is not as appropriate as a cost function for classification, given that the MSE makes assumptions about the ...
- 1,016
4
votes
Accepted
What are causative and exploratory attacks in Adversarial Machine Learning?
When someone is able to do a causative attack it means there is a mechanism by which they are able to input data into the network. Maybe a website where people can input their images and it outputs a ...
- 525
4
votes
Accepted
Why is there more than one way of calculating the accuracy?
In machine learning, the accuracy is usually defined as the number of correct predictions divided by the total number of predictions. The correct predictions are the true positives ($\mathrm {TP}$) ...
- 33.8k
4
votes
Accepted
Which paper introduced the term "softmax"?
The paper that appears to have introduced the term "softmax" is Training Stochastic Model Recognition Algorithms as Networks can Lead to Maximum Mutual Information Estimation of Parameters (...
- 33.8k
3
votes
What algorithm should I use to classify documents?
Text approach:
Use LDA (Latent Dirichlet Allocation). LDA is unsupervised. Feed it in corpuses of text from the various documents (i.e. OCR them and feed LDA the results of OCR). It will then cluster ...
- 76
3
votes
Is it possible to classify data using a genetic algorithm?
It is possible, but is a pretty terrible idea.
There are a few options. One is to not use the GA as a direct classifier, but instead use a GA to learn the parameters of another classification model ...
- 446
3
votes
3
votes
How to make convnets aware what the image actually is, not what is depicted on it?
You seem to be wanting some description of the 'style' of an image.
To make that work in general, I'd guess that would actually require quite a lot of pre-processing to present 'texture elements' (...
- 7,046
3
votes
Accepted
How successfully can convnets detect NSFW images?
The 2015 paper entitled "Applying deep learning to classify pornographic images and videos" applied various types of convnets for detecting pornography. The proposed architecture achieved 94.1% ...
- 1,999
3
votes
Do specific units exists for measuring the intelligence of a machine?
Shane Legg and Marcus Hutter proposed one in 2006. The main descriptive quotes (see the paper for the actual formula):
Intelligence measures an agent’s general ability to achieve goals in a wide ...
- 4,182
3
votes
Do specific units exists for measuring the intelligence of a machine?
One of the challenges of AI is defining Intelligence.
If we could precisely define general intelligence then we could program it into a computer. After all an algorithm is a process so well defined ...
- 554
Only top scored, non community-wiki answers of a minimum length are eligible